[cwn] Attn: Development Editor, Latest Caml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Apr 21 05:11:44 PDT 2009
Hello,
Here is the latest Caml Weekly News, for the week of April 14 to 21,
2009.
1) Amthing - a multi-threaded GUI library
2) ocamltarzan 0.1
3) OCaml-Java project: 1.2 release
4) "ok with parallel threads" GC (aka ocaml for multicore)
5) OSpec 0.2.0 - BDD for OCaml
========================================================================
1) Amthing - a multi-threaded GUI library
Archive: <http://groups.google.com/group/fa.caml/browse_thread/thread/af09a302fa8ab70e#
>
------------------------------------------------------------------------
** Continuing the thread from last week, David MENTRE asked and
Ogasawara Satoshi replied:
> 1. Are there any screenshots?
I've just take it.
<http://www.itpl.co.jp/amthing/concurrent_example.png>
> 2. Is there a widget system like in GTK/Qt?
Widgets are under developing. We will support button, label, textbox,
checkbox, radio button, tab, listbox, tree, grid,..etc. We already
supports
Pango and XIM for textbox.
> 3. What is the domain space covered by this library? Was it made for
> a specific purpose? IT Planning, Inc. is apparently a Japanese
company
> using OCaml but the main software is using Java Swing GUI.
The main purpose is to develop an ordinary desktop application covered
by GTK/Qt. In addition, Amthing is applicable to rich client which
access
to internet services. Animation makes application rich. Concurrent
environment
helps us to write asynchronous communications.
========================================================================
2) ocamltarzan 0.1
Archive: <http://groups.google.com/group/fa.caml/browse_thread/thread/7fee456d44ef9b02#
>
------------------------------------------------------------------------
** Yoann Padioleau announced:
This is the first release of ocamltarzan, a small fork of
Jane Street sexplib that some people may found more convenient
to use.
<http://aryx.kicks-ass.org/~pad/ocaml/ocamltarzan-0.1.tgz>
Motivations:
--------------------
Sexplib and binprot by Jane Street are attractive, but they rely on
camlp4. I don't like camlp4. I like the metaprogramming facility it
offers but it has many disadvantages. So I've found a in-the-middle
solution where I use camlp4 to generate code (via the small script
ocamltarzan.ml), save the generated code in a file (e.g
test/foo_sexp.ml), which allows then to completely remove the
dependency to camlp4. Once the code has been generated, all
dependencies to camlp4 can be removed. If tomorrow an incompatible new
version of camlp4 arrives (e.g. camlp6 ...), your code will _still_
work, because it does not rely on the new behavior of this camlp4.
It's just regular plain good ocaml code.
Example of use:
---------------
Given a file 'foo.ml' containing a type 't' which you would like to have
'sexp_of_t' and 't_of_sexp' functions, as well as 'sexp_of_tlist' and
'tlist_of_sexp', just add a comment annotation after the types as in:
type t = A | B
(* with sexp *)
type tlist = t list
(* with sexp *)
Then use my ocamltarzan (from its source directory) on this file
$ ./ocamltarzan tests/foo.ml > tests/foo_sexp.ml
The file foo_sexp.ml should now contain the 'xxx_of_sexp' and
'sexp_of_xxx' functions.
To use the new services offered by those functions, you can
write a use_foo.ml file such as:
let x = [Foo.A;Foo.A;Foo.B;Foo.A] in
let sexp = Foo_sexp.sexp_of_tlist x in
let s = Sexp.to_string_hum sexp (* 'hum' mean human readable *) in
print_string (s ^ "\n");
let chan = open_out "out.sexp" in
output_string chan s;
close_out chan;
let sexp2 = Sexp.load_sexp "out.sexp" in
let x2 = Foo_sexp.tlist_of_sexp sexp2 in
assert (x = x2);
()
This should lead to this output:
(A A B A)
Note that once foo_sexp.ml has been generated, the only thing
you really need to compile your code is the lib-sexp/ directory,
which as you can see is a plain regular good ocaml library,
with no camlp4 stuff involved.
Pro and cons of tarzan vs jane:
-------------------------------
pro:
- less camlp4
- less complicated to build
- arguably less complicated to use, e.g. no need for the
Type_conv_path stuff
- better control on the code generation as can easily customize
later the generated code, for instance
to not display certain things in sexp (like the cocci_tag,
position, etc)
- can provide a path for handling different version, an evolutionnary
format
- easier to debug when there is a problem ...
cons:
- have to regenerate when change code
- no Type_conv_path but have to do things manually with some module
aliases
- fragile if change order in .ml ?
========================================================================
3) OCaml-Java project: 1.2 release
Archive: <http://groups.google.com/group/fa.caml/browse_thread/thread/7c4f094a25acf0e0#
>
------------------------------------------------------------------------
** Xavier Clerc announced:
This post announces the 1.2 release of the OCaml-Java project.
The goal of the OCaml-Java project is to allow seamless integration of
OCaml
and Java.
Home page: <http://ocamljava.x9c.fr>
Download page: <http://ocamljava.x9c.fr/downloads.html>
Toplevel applet: <http://ocamljava.x9c.fr/toplevel/toplevel.html>
Main changes since 1.1:
- upgrade from version 3.10.2 to 3.11.0
- support for plugins ('native' dynlink)
- enhanced backtrace support
- enhanced support for camlp4
- experimental support for ocamlbuild
========================================================================
4) "ok with parallel threads" GC (aka ocaml for multicore)
Archive: <http://groups.google.com/group/fa.caml/browse_thread/thread/1a2410870ce4b725#
>
------------------------------------------------------------------------
** Philippe Wang announced:
Mathias and Adrien have just started their internship (for their
Master's degree requirements).
Thus they have some time to spend on this project. Moreover, Mathias'
internship is strongly related to this project.
=> man power dramatically increased
We are currently searching for the last remaining bugs.
Our thread library is restricted, it contains:
Thread : create, join, yield, id, self, delay
Mutex : full module
Condition : full module
Our alternative garbage collector
- uses a Stop(the world)&Copy algorithm
- has memory pages for threads (each thread takes a page at its
creation)
- has a shared heap for shared values and for old generation from
pages (i.e. memory pages are flushed to this heap)
- should be not to hard to replace.
Blocking sections such as I/O operations or mutex locks do not prevent
garbage collection.
We currently do *not* support POSIX signals (let's say their behaviour
is not specified).
We should make a release soon, but before:
- some code has to be cleaned
- some benchmarks have to be done
- some documentation has to be completed
- an installation script still has to be written.
Thus not a lot is left to do before the release :-)
We are writing test programs to search for the last remaining bugs but
also to measure performances.
So far, as long as there are not too many concurrent memory accesses,
it is not too hard to go n times faster with a n-core CPU;
though intense memory accesses generate page faults and divide memory
bandwidth by the number of concurrent accesses,
and intense memory consuming programs show our GC is not as performant
as INRIA's, of course.
** Xavier Clerc asked and Philippe Wang replied:
> Would it be correct to assume that the current state of the project
> implies that you have patched the OCaml runtime to make it
> fully reentrant ?
Is this following partial answer relevant enough ?
The garbage collector is clearly separated from the rest of the
runtime library: the GC is contained in a libgc.a and our patched
ocamlopt links programs to this GC. The GC variables are known by the
GC only.
> If so, is this code/patch available for download ?
Officially, not yet (and not before April 20th).
We did not expect the debugging part to be so complex and hard, and
taking so long.
The man power dramatically decreased in late September : the 2
Master's students went back to Master's courses, and the 3 supervisors
had to do research in parallel with teaching.
Some major bug fixes were made in February/March, a lot of major bug
fixes were made in April (yes, these last 2 weeks).
You know, bugs hiding other bugs... however we are hopefully getting
close to the fix point: today there is no known bug ! :-)
Unsupported features are - of course - not considered as bugs. For
instance, posix signals are (currently) not supported. And, as
parallel computing *potentially* requires quite a lot more memory,
some programs can easily end up in a blocked state when the heap
becomes full: our GC (currently) uses (parameterized) fixed size pages
and heap.
The next days, we will concentrate on making benchmarks (if you have
some relevant testing programs, they are welcome), and if we don't
discover new bugs then we will focus on (finishing) writing a
documentation and a building script, for the release.
If we release as such now, we will have too much support to do because
of the lack of documentation. So it's not quite a good idea...
When we have the minimal-but-sufficient documentation, we will make
the release :-)
In parallel, we try to make it work with OS X Leopard 64 bit and/or
ocaml 3.11 (currently we only support 3.10.2 - Linux x86_64).
** Xavier Clerc asked and Philippe Wang replied:
> Is your implementation still based on a global runtime lock ?
No, it isn't. (And it would probably too often prevent parallel threads,
wouldn't it.)
> A negative answer would imply that you patched the OCaml
> runtime to make it reentrant. To illustrate my point, I will take
> the example of the file "byterun/compare.c". In this file, the
> code for the comparison of values makes use of a global
> variable (named "caml_compare_unordered").
It should be, unless bugs are still hiding under that.
It is supposed to have been done for a while. We'll make one more check.
> If you patched the runtime to allow multiple threads to use
> it concurrently, I would also be interested by the strategy
> used: is the problem solved by additional parameters ?
> by the use of thread-local storage ? by any other mean ?
- local variable instead of global variable in functions
- some functions are parameterized by thread identifier (that is, one
more
parameter than "before") (e.g. in amd64.S)
** Philippe Wang later added:
Well, we went back into runtime code implementation.
This is what can be said rapidly :
- compare.c contains no global variables anymore, we use local variables
instead
- if a Caml-C or C-Caml interface uses caml_compare_unordered, we
don't know
what can happen with parallel threads
- we have many global mutex locks with small scopes
- we do use an "enter/leave blocking section" mechanism to prevent the
GC from
waiting on a blocking operation such as I/O or mutex locking etc.
- we don't support weak values (not sure whether they don't work or they
became strong, if they dont work anymore, they can be back in 2
minutes as
strong values anyway)
- serialisation of values is a little bit tricky, though it should work
- most important : many global variables do not exist anymore because
they are
irrelevant in our implementation
- we do not support unofficial-features of ocaml 3.10, e.g. the new
features
that come with 3.11 but actually have their roots in previous
versions
~ it is almost sad to see all the based-on-"one-thread-at-a-time"
optimisations removed...
+ (it looks like it works just fine)
I hope there are no "hidden" bad global variables.
Is it fully reentrant ? Hmmmm... maybe.
We use a stop-the-world GC (which means no one is running is parallel
with the
GC), that is actually like original ocaml, that comes with its
inconveniences
: C calls not declared as blocking sections (which has quite a cost) may
prevent other threads from running when the heap is full.
Graphics module, for instance, is not reentrant at all (anyhow it's
not part
of the runtime). Same for Str.
Funny thing is we can open several windows by launching parallel threads
(though only one is useful at the end).
Anyway, thank you for your questions and interest, they have helped us
find&fix some bugs.
========================================================================
5) OSpec 0.2.0 - BDD for OCaml
Archive: <http://groups.google.com/group/fa.caml/browse_thread/thread/39bd2c37dcb9f986#
>
------------------------------------------------------------------------
** Andre Nathan announced:
I'm happy to announce a new version of OSpec, a Behavior-Driven
Development
tool for OCaml using a Camlp4 syntax extension. You can download this
release
from the ocamlcore forge at
<http://forge.ocamlcore.org/projects/ospec/>
or directly clone the repository from Github:
<http://github.com/andrenth/ospec/tree/master>
== New features:
* Nested specifications: it is now possible to group related examples
in a
nested "describe" block. For example,
describe "Person" do
describe "name" do
it "should not be empty" do
(String.length person.name) should be > 0
done;
it "should only contain valid characters" do
(person.name) should match_regexp "^[A-Za-z- ]+$"
done
done
done
* Property testing (inspired by QuickCheck): OSpec can now autogenerate
test cases with the "forall" keyword. For example,
describe "A list" do
it "should equal itself when reversed twice" do
forall list l . (List.rev (List.rev l)) should = l
done
done
This will generate lists of random length and test each one against the
specified property.
There are 28 predefined random sample generators for the basic OCaml
types which can be used directly in tests or as building blocks for
custom generators.
* Better error handling: exceptions in the specifications are now caught
and reported.
== Bug fixes:
* Properly count pending examples.
OSpec is released under the MIT license. Please see the README file in
the
distribution for more details and examples.
========================================================================
Using folding to read the cwn in vim 6+
------------------------------------------------------------------------
Here is a quick trick to help you read this CWN if you are viewing it
using
vim (version 6 or greater).
:set foldmethod=expr
:set foldexpr=getline(v:lnum)=~'^=\\{78}$'?'<1':1
zM
If you know of a better way, please let me know.
========================================================================
Old cwn
------------------------------------------------------------------------
If you happen to miss a CWN, you can send me a message
(alan.schmitt at polytechnique.org) and I'll mail it to you, or go take a
look at
the archive (<http://alan.petitepomme.net/cwn/>) or the RSS feed of the
archives (<http://alan.petitepomme.net/cwn/cwn.rss>). If you also wish
to receive it every week by mail, you may subscribe online at
<http://lists.idyll.org/listinfo/caml-news-weekly/> .
========================================================================
--
Alan Schmitt <http://alan.petitepomme.net/>
The hacker: someone who figured things out and made something cool
happen.
.O.
..O
OOO
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20090421/56b0bdb3/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
Url : http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20090421/56b0bdb3/attachment-0001.pgp
More information about the caml-news-weekly
mailing list