[LWN Logo]
[Timeline]
From: Brian Ingerson <brian@ingerson.com>
Newsgroups: comp.lang.perl.announce,comp.lang.perl.modules
Subject: ANNOUNCE: Inline 0.26 "You got your *C* in my *Perl* code!"
Date: 27 Sep 2000 07:26:43 GMT

0.26 shields Inline.pm from its users changing the $/ and $\ globals.
This allows you to use the '-l' in Perl one-liners such as:

perl -lwe 'print add(3,4);use Inline C=>"int add(int x,int y){return
x+y;}"'

Thanks to Michael Schwern for pointing out the bug.

Also, Inline is now a perl6 RFC. #270. Write your congressman.

---

0.30 will be the next major release of Inline, due out soon after I
finish moving to Vancouver. It will include:

- All C data types will now come from typemap files. The default Perl
typemap file contains all of the current types included in Inline 0.26
(int, long, double, char*, SV*) plus many more. Will also provide easy
integration of existing and new typemap files. Extensibility and XS
backwards compatability to the max!

- Syntax for interacting easily with other modules that provide a C API.
Something like "use Inline with => 'Event';". This allows the Event
module to provide compilation hints to the Inline module. (I'm working
with Event.pm author, Joshua Pritikin, for initial test case.)

- Support for a new Inline code marker token: '__C__'. This will allow
different types of information to peacefully coexist in the DATA
section, like: Inline, POD, and AutoLoader subroutines.

- Support for my new programming language, CPR. :-)

---

0.35 is but a gleam in my eye, but will probably include:

- Support for distributing pre-compiled extensions for platforms lacking
compilers. Would work with ActiveState's PPM.

- Enhanced debugging features.

- Robust test harness.

- More doc and tons of sample code.

---

As usual, here's the current doc in text format:

NAME
    Inline - Use other programming languages inside Perl scripts and
    modules.

SYNOPSIS
        print "9 + 16 = ", add(9, 16), "\n";
        print "9 - 16 = ", subtract(9, 16), "\n";
     
        use Inline C => <<'END_OF_C_CODE';
        
        int add(int x, int y) {
          return x + y;
        }
     
        int subtract(int x, int y) {
          return x - y;
        }
       
        END_OF_C_CODE

DESCRIPTION
    The `Inline' module allows you to put source code from other
    programming languages directly "inline" in a Perl script or
    module. The code is automatically compiled as needed, and then
    loaded for immediate access from Perl.

    `Inline' saves you from the hassle of having to write and
    compile your own glue code using facilities like XS or SWIG.
    Simply type the code where you want it and run your Perl as
    normal. All the hairy details are handled for you. The
    compilation and installation of your code chunks all happen
    transparently; all you will notice is the delay of compilation.

    The `Inline' code only gets compiled the first time you run it
    (or whenever it is modified) so you only take the performance
    hit once. Code that is Inlined into distributed modules (like on
    the CPAN) will get compiled when the module is installed, so the
    end user will never notice the compilation time.

    Best of all, it works the same on both Unix and Microsoft
    Windows. See the section on "SUPPORTED PLATFORMS" below.

  Why Inline?

    Do you want to know "Why would I use other languages in Perl?"
    or "Why should I use `Inline' to do it?"? I'll try to answer
    both.

    Why would I use other languages in Perl?
        The most obvious reason is performance. For an interpreted
        language, Perl is very fast. Many people will say "Anything
        Perl can do, C can do faster". (They never mention the
        development time ;-) Anyway, you may be able to remove a
        bottleneck in your Perl code by using another language,
        without having to write the entire program in that language.
        This keeps your overall development time down, because
        you're using Perl for all of the non-critical code.

        Another reason is to access functionality from existing API-
        s that use the language. Some of this code may only be
        available in binary form. But by creating small subroutines
        in the native language, you can "glue" existing libraries to
        your Perl. As a user of the CPAN, you know that code reuse
        is a good thing. So why throw away those Fortran libraries
        just yet?

        If you are using Inline with the C language (currently the
        only way to do it), then you can access the full internals
        of Perl itself. This opens up the floodgates to both extreme
        power and peril.

        Maybe the best reason is "Because you want to!". Diversity
        keeps the world interesting. TMTOWTDI!

    Why should I use `Inline' to do it?
        There are already two major facilities for extending Perl
        with C. They are XS and SWIG. Now if you're familiar with
        either, then I may be preaching to the choir. Well, here
        goes:

         <SERMON>

        Greetings congregation. This morning I want to open your
        eyes to the virtues of Inline and the perils of XS. Let us
        compare the two.

        ---

        Inline - You can use it from a regular script.

        XS - Requires you to create a module and an XS file and a
        makefile, in addition to your regular script. Actually, the
        program `h2xs' does a nice job of getting you started, but
        that's still a lot of junk to maintain.

        ---

        XS - You need rebuild every time you want to test a small
        change.

        Inline - Perl programmers cannot be bothered with silly
        things like compiling. "Tweak, Run, Tweak, Run" is our way
        of life. `Inline' does all the dirty work for you.

        ---

        XS - There is a difficult learning curve involved with
        setting up and using the XS environment. (At least for a
        simple Perl preacher like me.) Read the following perldocs
        and man pages if you don't believe me:

         * perlxs
         * perlxstut
         * perlapi
         * perlguts
         * perlmod
         * h2xs
         * xsubpp
         * ExtUtils::MakeMaker

        Inline - Makes easy things easy, and hard things possible.
        Just like Perl.

        ---

        XS - Only implements C and C++.

        Inline - Plans to implement several languages. For now,
        `Inline' only implements C and it uses XS to do it. (Dirty
        little secret) But this is the right thing to do. See the
        section on "SUPPORTED LANGUAGES" below.

        ---

        Amen.

         </SERMON>

  How it works

    `Inline' performs the following steps:

    1) Receive the Source Code
        `Inline' gets the source code from your script or module
        with a statement like the following:

         use Inline C => Source-Code;

        where `C' is the programming language of the source code,
        and `Source-Code' is a string (most easily represented by
        using the "Here Document" quoting style; see the section on
        "SYNOPSIS" above), a file name, an open file handle, or a
        reference to a subroutine (that will return source code).

        Since `Inline' is coded in a "`use'" statement, everything
        is done during Perl's compile time. If anything needs to be
        done that will affect the `Source-Code' string, it needs to
        be done in a `BEGIN' block that is *before* the "`use Inline
        ...'" statement. This might include setting interpolated
        variables, or setting options in the `Inline::Config'
        module.

    2) Check if the Source Code has been Compiled
        `Inline' only needs to compile the source code if it has not
        yet been compiled. It accomplishes this seemingly magical
        task in an extremely simple and straightforward manner. It
        runs the source text through the `Digest::MD5' module to
        produce a 128-bit "fingerprint" which is virtually unique.
        The fingerprint (in hex) is *mangled* with the current
        package name (and the script name, if the package is
        "`main'") along with the name of the programming language,
        to form a unique name for the executable module. For
        instance, the `C' code from `examples/example001.pl' (see
        the section on "Examples In C") would mangle into:

         main_C_example001_pl_3a9a7ba88a8fb10714be625de5e701f1.so

        If an executable with that name already exists, then proceed
        to step 8. (No compilation is necessary)

    3) Find a Place to Build and Install
        At this point `Inline' knows it needs to compile the source
        code. The first thing to figure out is where to create the
        great big mess associated with compilation, and where to put
        the object when it's done.

        By default `Inline' will try to build and install under the
        first one of the following places that is a valid directory
        and is writable:

             1) $ENV{PERL_INLINE_BLIB}
                (The PERL_INLINE_BLIB environment variable overrides all
else)
             2) ./blib_I/  
                (The current directory, unless you're in your home
directory)
             3) $bin/blib_I/
                (Where '$bin' is the directory that the script is in)
             4) $ENV{HOME}/blib_I/
                (Under your home directory)
             5) $ENV{HOME}/.blib_I/
                (Same as above but more discrete)

        If none of those exist, Inline will attempt to create and
        use one of following:

             6) $bin/blib_I/ 
             7) ./blib_I/ 

        Failing that, Inline will croak. Optionally, you can
        configure `Inline' to build and install exactly where you
        want, using `Inline::Config'. See the Inline::Config
        manpage. If `$Inline::Config::SITE_INSTALL=1', then `Inline'
        will only use `./blib_I/' to build in, and the
        `$Config{installsitearch}' directory to install the
        executable in. This option is intended to be used in modules
        that are to be distributed on the CPAN, so that they get
        installed in the proper place.

        Optionally, you can configure `Inline' to build and install
        exactly where you want.

        NOTE: `blib' stands for "build library" in Perl-speak. It is
        a directory that gets created when you install a Perl module
        on your system. `blib_I' is the `Inline.pm' version of the
        same concept.

    4) Parse the Source for Semantic Cues
        `Inline' uses the module `Parse::RecDescent' to parse
        through your chunks of source code and look for things that
        it can create run-time bindings to. For instance, in `C' it
        looks for all of the function definitions and breaks them
        down into names and data types. These elements are used to
        correctly bind the `C' function to a `Perl' subroutine.

    5) Create the Build Environment
        Now `Inline' can take all of the gathered information and
        create an environment to build your source code into an
        executable. Without going into all the details, it just
        creates the appropriate directories, creates the appropriate
        source files including an XS file and a `Makefile.PL'.

    6) Compile the Code and Install the Executable
        The planets are in alignment. Now for the easy part.
        `Inline' just does what you would do to install a module.
        "`perl Makefile.PL && make && make test && make install'".
        If something goes awry, `Inline' will croak with a message
        indicating where to look for more info.

    7) Tidy Up
        By default, `Inline' will remove all of the mess created by
        the build process, assuming that everything worked. If the
        compile fails, `Inline' will leave everything intact, so
        that you can debug your errors. Setting
        `$Inline::Config::CLEAN_AFTER_BUILD=0' will also stop
        `Inline' from cleaning up.

    8) DynaLoad the Executable
        `Inline' uses the `DynaLoader::bootstrap' method to pull
        your external module into `Perl' space. Now you can call all
        of your external functions like Perl subroutines. Wheeee!

  Another Way To Do It

    Instead of specifying the source code as a here-document string,
    you may want to put it at the end of your script, after the
    `__END__' statement. Then you can pass it to `Inline' using the
    `DATA' filehandle, like this:

        use Inline;
        Inline->import(C => <DATA>);
        
        print "9 + 16 = ", add(9, 16), "\n";
        print "9 - 16 = ", subtract(9, 16), "\n";
        
        __END__
        
        int add(int x, int y) {
          return x + y;
        }
        
        int subtract(int x, int y) {
          return x - y;
        }

    Since the `use' command is executed at compile time and the
    `DATA' filehandle isn't available until runtime, you'll need to
    invoke the `import' call manually. This is a useful idiom
    anytime you need to specify `Inline' code at runtime.

  Configuration

    `Inline' trys to do the right thing as often as possible. But
    sometimes you may need to override the default actions. This is
    where `Inline::Config' comes to the rescue. `Inline::Config'
    gives you a more fine-grained control over the entire process.
    The other side of that coin is "you need to know what you are
    doing".

    An important point to remember is that the config settings must
    be done *before* the `Inline' code is evaluated. Since a "`use'"
    happens at (`Perl''s) compile time, you may need to something
    like this:

        BEGIN {
            use Inline;
            $Inline::Config::OPTION_NUMBER_9 = 'Yes';
        # or
            Inline::Config->new->option_number_9('Yes');
        }
        
        use Inline C => "C code goes here...";

    See the Inline::Config manpage for more info.

  Configuration from the Command Line

    `Inline' lets you set many of the configuration options from the
    command line. This can be very handy, especially when you only
    want to set the options temporarily, for say, debugging.

    For instance, to get some general information about your
    `Inline' code in the script `Foo.pl', use the command:

        perl -MInline=INFO Foo.pl

    If you want to force your code to compile, even if its already
    done, use:

        perl -MInline=FORCE Foo.pl

    If you want to do both, use:

        perl -MInline=INFO -MInline=FORCE Foo.pl

    or better yet:

        perl -MInline=INFO,FORCE Foo.pl

    See the Inline::Config manpage for more info.

  Writing Modules with Inline

    Writing CPAN modules that use other programming languages is
    easy with `Inline'. Let's say that you wanted to write a module
    called `Math::Simple' using the previous example code. Start by
    using the following command:

        h2xs -PAXn Math::Simple

    This will generate a bunch of files that form a skeleton of what
    you need for a distributable module. Next, modify the
    `Simple.pm' file to look like this:

        package Math::Simple;
        
        use strict;
        use vars qw($VERSION @ISA @EXPORT_OK);
        require Exporter;
        @ISA = qw(Exporter);
        @EXPORT_OK = qw(add subtract);
        BEGIN {
            $VERSION = '0.01';
        }
        
        use Inline;
        Inline->import(C => <DATA>);
        
        1;
        
        __DATA__
        
        int add(int x, int y) {
          return x + y;
        }
        
        int subtract(int x, int y) {
          return x - y;
        }

    Finally, you need to add the following line to the top of your
    `test.pl' file:

        use Inline SITE_INSTALL;

    When the person installing `Math::Simple' does a "`make test'",
    the `Inline' module will compile the Inlined code and place the
    executable code into the `./blib' directory. Then when a "`make
    install'" is done, the module will be copied into Perl's
    `$Config{installsitearch}' directory (which is where an
    installed module should go).

    Now all you need to do is:

        perl Makefile.PL
        make dist

    That will generate the file `Math-Simple-0.01.tar.gz' which is a
    distributable package.

  Fancy Tricks

    The `Inline' module opens up all sorts of possibilities
    regarding what you can do with `Perl' and `C'. Since everything
    happens at run time (depending on how you think of it) you can
    generate `C' code on the fly and effectively '`eval'' it. (How
    this might be useful is left as an exercise to the reader :-)

    Here is how you would code such a beast:

        BEGIN {$c_code = &c_code_generator()}
        use Inline C => $c_code;  # will die if code doesn't compile
        my_function();

    or

        $c_code = &c_code_generator();
        eval {use Inline C => $c_code};
        if ($@) {
            handle_error($@);     # trap error if code doesn't compile
        }
        else {
            my_function();
        }

SUPPORTED LANGUAGES
    Currently, "`C'" is the only supported language. This is
    obviously the most important language to support. That is
    because `Perl' itself is written in `C'. By giving a your `Perl'
    scripts access to `C', you in effect give them access to the
    entire glorious internals of `Perl'. (Caveat scriptor :-)

    `C' is also the easiest language to implement because the tools
    needed to do so, (like XS and `ExtUtils::MakeMaker') have
    already been written and are very flexible and reliable.
    `Inline' currently makes use of these pre-existing tools.

    But there is definitely no reason why `Inline' must or should
    stop with `C'. As long as sensible bindings can be defined
    between Perl and another language, that language could be a
    candidate for the `Inline' module. Current languages I am
    considering adding support for include:

     - C++
     - Fortran
     - Pascal
     - Python

    Note: Since many `C' compilers allow the use of assembly code
    within C, you may want to consider Assembly Language as
    supported. Ready to start scripting out new device drivers?

SUPPORTED PLATFORMS
    This module should work anywhere that CPAN extension modules
    (those that use XS) can be installed, using the typical install
    format of:

        perl Makefile.PL
        make
        make test
        make install

    It has been tested on many Unix variants and Windows NT.

    NOTE: `Inline.pm' requires Perl 5.005 or higher because
    `Parse::RecDescent' requires it. (Something to do with the `qr'
    operator)

    Inline has been tested on the following platforms:

     V#   OS      OS V#   Perl V# Human              Email 
     0.25 Linux   2.2.13  5.00503 Brian Ingerson     ingy@cpan.org
     0.25 Linux   2.2.13  5.6     Brian Ingerson     ingy@cpan.org
     0.20 FreeBSD 3.4     5.00503 Timothy A Gregory 
tgregory@tarjema.com      
     0.20 FreeBSD 4.0     5.00503 Timothy A Gregory 
tgregory@tarjema.com      
     0.20 FreeBSD 4.0     5.6     Timothy A Gregory 
tgregory@tarjema.com      
     0.20 Linux   2.0.36  5.00503 Prakasa Bellam    
pbellam@cobaltgroup.com
     0.20 HPUX    B.10.20 5.00503 Jamie Shaffer     
jshaffer@chronology.com
     0.20 SunOS   5.6     5.6.0   Jamie Shaffer     
jshaffer@chronology.com
     0.20 SunOS   5.5.1   5.6.0   Jamie Shaffer     
jshaffer@chronology.com
     0.22 OpenBSD 2.7     5.6.0   Jeremy Devenport   jeremy@weezel.com
     0.22 FreeBSD 3.1     5.00503 Doug Beaver        dougb@scalar.org
     0.25 WinNT   4.0 sp6 5.00503 Brian Ingerson     ingy@cpan.org
     0.24 Cygwin  1.1.1   5.6.0   Leo Schalkwyk     
L.Schalkwyk@iop.kcl.ac.uk

    The Microsoft tests deserve a little more explanation. I used
    the following:

     Windows NT 4.0 (service pack 6)
     Perl 5.005_03 (ActiveState build 522)
     MS Visual C++ 6.0
     The "nmake" make utility (distributed w/ Visual C++)

    `Inline.pm' pulls all of its base configuration (including which
    `make' utility to use) from `config.pm'. Since your MSWin32
    version of Perl probably came from ActiveState (as a binary
    distribution) the `Config.pm' will indicate that `nmake' is the
    system's `make' utility. That is because ActiveState uses Visual
    C++ to compile Perl.

    To install `Inline.pm' (or any other CPAN module) on MSWin32 w/
    Visual C++, use these:

        perl Makefile.PL
        nmake
        nmake test
        nmake install

    The "Cygwin" test was done on a Windows 98 machine using the
    Cygwin Unix/Win32 porting layer software from Cygnus. The `perl'
    binary on this machine was also compiled using the Cygwin tool
    set (`gcc'). This software is freely available from
    http://sources.redhat.com/cygwin/

    If `Inline' works on your platform, please email me the info
    above. If it doesn't work, see the section on "BUGS AND
    DEFICIENCIES" below.

SEE ALSO
    the Inline::Config manpage and the Inline::C::Tutorial manpage

BUGS AND DEFICIENCIES
    This is ALPHA code. The interface may still change.

    When reporting a bug, please do the following:

     - Put "use Inline REPORTBUG;" at the top of your code, or
       use the command line option "perl -MInline=REPORTBUG ...".
     - Run your code.
     - Follow the printed directions.

    Here are some things to watch out for:

    1   The `Parse::RecDescent' grammar for `C' is fledgling. It'll get
        better. For now be careful and examine the generated code
        when things don't work. Also, using "`perl -MInline=INFO
        ...'" will give you useful information.

    2   `Inline' doesn't yet support custom typemapping. To pass
        anything beyond basic C types, use the type `SV*' and do
        your own conversions "inline". See the Inline::C::Tutorial
        manpage for more information on programming with C.

    3   While `Inline' does attempt to clean up after itself, there is
        currently no functionality to remove a shared object when a
        new version is compiled. This shouldn't be hard to do, but I
        want to think about it a little more.

    4   The compile time using Visual C++ on MSWin32 seems to be much
        slower in my tests so far. During this time, your script
        will seem to hang. Just be patient. After compilation, the
        execution time is comparable.

AUTHOR
    Brian Ingerson <INGY@cpan.org>

COPYRIGHT
    Copyright (c) 2000, Brian Ingerson. All Rights Reserved. This
    module is free software. It may be used, redistributed and/or
    modified under the terms of the Perl Artistic License.

    (see http://www.perl.com/perl/misc/Artistic.html)