From: Alan Schmitt <alan.schmitt@inria.fr> To: lwn@lwn.net Subject: Attn: Development Editor, Latest Caml Weekly News Date: Tue, 21 May 2002 17:25:31 +0200 Hello, Here is the latest Caml Weekly News, weeks 14 to 21 may, 2002. 1) Generating C stubs 2) COM binding & CAMLIDL ? 3) Dynamic Caml v0.2 4) Surreal-0.0.3 5) OCaml packaging problems: findlib helps! ====================================================================== 1) Generating C stubs ---------------------------------------------------------------------- Jérôme Marant asked: Hi, In order to avoid spending some time on writing C stubs myself, I'd like them to be generated automaticaly. There are currently 3 stub generators: camouflage, ffi and another one I don't recall (IIRC, someone claimed that he was working on a SWIG interface). I'm a bit confused since I don't really know what generator to choose among all those. Moreover, they all seem to have been last update in august 2001 and the C interface may have changed since. Has anyone experiences and advices about this? Thanks in advance. Jocelyn Sérot answered: I've been using CAMLIDL (http://caml.inria.fr/camlidl/) extensively and it works well. It does not work directly from your C headers (.h) but from a IDL signature, but writting this sig is straightforward in most of the cases. Benedikt Grundmann also saidi (about a SWIG interface): That was me and I'm still working on it. I just started working on automatic structure wrapping. This project (ocaml-swig) is the first time I started working on the core of swig which unfortunately is constantly changing because Dave and the others trying to improve the C++ support. So I really can't tell you when a working release of the ocaml-swig module will be done. To which Jeff Henrikson added: No it was me! ;-) Actually I basically did write a SWIG language module. (How is it that we didn't see each others work on the list archives?) Anyway, after writing about 2000 lines of C code but before getting the bugs worked out, I abandoned it. My reasoning was: 1) SWIG is not sophisticated enough to get the behavior I wanted easily. 2) Doing extra work to approach what I wanted was making massive code bloat (SWIG is written in C and is not forgiving of changing the "normal" behavior from the language module end) 3) I hate programming in C, especially when the API has been constructed to treat _everything_ as a void*. 4) Someone released a binding for the interfaces I needed So that put the lid on that one for a while. But I learned why exactly I find it (and the IDL approach, and other FFI systems) inadequate. SWIG tries to be a little bit smarter than IDL by letting you break away from the "this parameter is [in], this parameter is [out]" level of abstraction, by allowing some weak sorts of pattern matching. In other words, it can take the names of the parameters in a C header file and say "make all int parameters named 'point' and convert them this way. But make all the int parameters named 'color' and convert them this other way." But it's not general enough, and still makes the mistake of annotating the header file rather than making the annotations separate. So when version 2.0 of your header files come out, you start from scratch or do some silly diff3 mucking around. I started implementing a new header generator based on the idea of describing the annotations as a caml program. Basically have a bunch of convienience functions that search through a C abstract syntax tree to ask things like this: 1) I need to use function FooOp which takes a datatype Foo*. Read mydefs.h which contains their prototypes into the current environment. 2) Find me all descendant types which are needed to build the struct Foo. 3) Generate me constructor functions to build these ones which I declare to be public members, and any of their children. For primitive types, convert using these C code snippets I give you (ala SWIG typemaps). 4) Find me all functions which take a Foo* as a first parameter. Write C wrappers for them, caml external declarations, and put them together as a class in an OOP wrapper ala lablgtk. Truncate the method names from Foo_happyOperation to just myFoo#happyOperation by a regular expression substitution, with exceptions which I specify by hand. Anyway, needless to say there's a bunch of stuff to be worked out, but I may get back at the project soon, as I am needing another API binding. I really want to short circuit the writing each function by hand as you go thing. The problem is that I usually work with APIs that I don't know upfront, and want to just generate the stubs in bulk and learn the API straight from an ML environment. (This is one of the areas where caml is definitely not as easy as C yet.) I think 95% of the necessary information is already contained in well stylized header files, we just need a way of systematically cleaning up the last 5% into a complete IDL level of knowledge. Two further extensions of such a system would be to 1) produce a language independent output, in other words write out IDL instead of C wrappers. 2) construct a language independent input, so that non-caml programmers might be seduced. :-> Then Jeff Henrikson made the code available: I stuck it on the web for you. Don't laugh, it's the first thing on my new iDisk. http://homepage.mac.com/jehenrik/ ====================================================================== 2) COM binding & CAMLIDL ? ---------------------------------------------------------------------- Samuel Lacas asked and Dmitry Bely answered: > Reading a recent thread on the list, I noticed that ocaml could be > used with COM components on windows. As I currently intensively use > the Python COM bindings with MS' text processor whose name I won't > say, I wonder if I could use ocaml to the same end. Probably yes, if you are going to use/create COM components with vtbl-style interfaces. Support for IDispatch interfaces is very limited (see camlidl manual). > However, the release notes in the ocaml-win distribution states that > the cygwin version lacks COM support, In fact Cygwin gcc has all necessary COM libraries and camlidl can be easily patched to support COM interfaces for Cygwin-build OCaml (I have done that if you are interested). > and the native win32 version > partially implements the libraries, among which I find no COM nor IDL > link. Does anyone on the list have more information ? [ocaml\README.win32] The native Win32 port has threads and COM support [camlidl\readme] In addition, Camlidl provides basic support for COM interfaces and components under MS Windows. It supports both using COM components (usually written in C++ or C) from Caml programs, and packaging Caml objects as COM components that can then be used from C++ or C. [end of quote] So yes, MSVC-compiled OCaml supports Camlidl and COM out-of-the-box. ====================================================================== 3) Dynamic Caml v0.2 ---------------------------------------------------------------------- Dmitry Lomov announced: I am pleased to announce the availability of Dynamic Caml v.0.2 from: http://oops.tercom.ru/dml/ Dynamic Caml is a high-level, type-safe (statically typed) and efficient dynamic code generation library and a set of CamlP4 syntax extensions for Objective Caml. Though still supporting only bytecode, release 0.2 is much more mature than the previous 0.1 release. This is the list of changes: - Most of Objective Caml expressions are supported either natively (as constructors of code type) or via functional wrappers that are automatically generated by syntax extensions. Not supported expressions are: * labelled applications * OO features (method calls and such) Those can be used via manually written functional wrappers. - Functional wrappers for expressions that are not natively supported are inlined in the generated bytecode, thus removing application overhead. - Several bugfixes - More examples added (see directory tests/ in the distribution) Your comments and suggestions are welcome! ====================================================================== 4) Surreal-0.0.3 ---------------------------------------------------------------------- Yamagata Yoriyuki announced: Surreal-0.0.3 is released. Surreal library is another exact real arithmetic library for objective caml. The library is currently just a rough sketch, and not well tested. There is no plan to further development. I am tired of error bound estimation. However, if someone is interested, I would love to share ideas. For more info, look to the home page. http://www.mars.sphere.ne.jp/yoriyuki/surreal.html I would appreciate your feedbacks ====================================================================== 5) OCaml packaging problems: findlib helps! ---------------------------------------------------------------------- Gerd Stolpmann announced: I have prepared an experimental version of findlib (download see below) that allows users to store DLLs into a shared directory, called "libexec". As you may know, findlib organizes packages by directory. For example, an installation with three packages X, Y, and Z could look like: /usr/local/lib/ocaml/site-lib/X/<files> /usr/local/lib/ocaml/site-lib/Y/<files> /usr/local/lib/ocaml/site-lib/Z/<files> Previous versions of findlib put DLLs simply into the package directories, and findlib tried to update the ld.conf file (which may fail because of missing file permissions). A similar solution is also used by the Debian packagers. There are some problems with it: - There are scalability problems when really many directories are in ld.conf - Package management systems like dpkg and rpm must use postinstall scripts which makes packaging more difficult - There is a conflict between dpkg/rpm packages and additional manual installations: Who is responsible for which line in ld.conf? - Name clashes are hard to detect (e.g. X and Y use both dllfoo.so) The experimental version of findlib recognizes a special directory libexec: /usr/local/lib/ocaml/site-lib/libexec/<DLL files> The simple existence of this directory causes that all files with the suffixes .so and .dll are copied into this directory (no symlinks). Of course, this breaks the current way of representing the ownership of files. In previous versions, all files in the package directory for X are owned by X. This is simple, and works for everything else very well. In libexec, there are now .owner files, e.g. for dllfoo.so findlib creates a second file dllfoo.so.owner, containing only one line with the name of the owner. Knowing the owner of files is necessary to delete a package. The advantages of this solution are: - It is not necessary to update ld.conf when a package is installed or deleted. Usually, there are very few such libexec directories, and they are known in advance. - dpkg and rpm do no longer need postinstall scripts. - The lookup of DLLs is faster. - findlib checks whether there are name clashes. Nothing is perfect: - Recognizing a DLL by its file extension may be wrong - The .owner files are ugly, and this functionality overlaps with dpkg/rpm. - It is still difficult to install several versions of the same library. Although findlib supports several package repositories, this does not help, because all libexec directories must be mentioned in ld.conf, and the order in this file determines which library is loaded. The workaround is to set CAML_LD_LIBRARY_PATH to influence the search order manually. If you want to look at this experiment, you can download it from http://www.ocaml-programming.de/packages/findlib-0.7-pre.tar.gz ====================================================================== Alan Schmitt -- The hacker: someone who figured things out and made something cool happen.