[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Aug 30 02:52:26 PDT 2022
Hello
Here is the latest OCaml Weekly News, for the week of August 23 to 30,
2022.
Table of Contents
─────────────────
What’s your development workflow?
Mirage retreat October 3rd - 9th
Experience report: making a JavaScript library from an OCaml library with js_of_ocaml
Will I ever understand Format? or How to print a list vertically with indentation
GNU Guile 1.0.0 - Bindings to GNU Guile for OCaml
Update on Eio (effects-based direct-style IO for OCaml 5)
OCaml at First Glance
GADTs for typed option getter/setter
shuttle v0.3.1 released
coinductive data types
Old CWN
What’s your development workflow?
═════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/whats-your-development-workflow/10358/1>
Bozhidar Batsov asked
─────────────────────
I’ve started learning OCaml recently and one issue that I struggle
with is finding a workflow that gives me quick feedback about changes
I’ve made (I’m looking to reduce compile/run cycles). To give you some
context - most of the time I’m programming in Lisps and Ruby, where
it’s relatively easy to modify a running application. With all Lists
my workflow is:
• I make some change (e.g. I update some function)
• I compile the changed bit of code (e.g. a function)
• I immediately run this with the help of the REPL
• Based on the results I got I go to the beginning or move on to the
next things that needs doing
That’s known as [interactive programming] in some circles and I
totally love it. With Ruby I can’t do the same, but at least with web
apps there’s some degree of code reloading that allows you modify a
running app.
With OCaml, however, I’m not sure how to get the quick feedback, as it
seems I need to constantly compile and run stuff. I’m trying to work
like with Lisps - I keep a toplevel open alongside my source code and
occasionally send some code there, but the toplevel is limited in many
way (you’re just dumping text there at the global level). `dune utop`
helps to some extent, but it can’t reload changes. I’ve heard that
some people were saying they didn’t even use a toplevel, so I’m
wondering if I’m missing something. I’d be curious to hear any tips
and tricks from your OCaml workflows.
[interactive programming]
<https://docs.cider.mx/cider/usage/interactive_programming.html>
Jack Feser suggested
────────────────────
Have a look at expectation testing with [ppx_expect]. That’s what I
use to get this kind of quick feedback (and you can keep the results
as a test).
Instead of sending code to a REPL, you write your test in an expect
block and run `dune runtest'. Anything you print out in the expect
gets captured, and dune shows you the output as a diff.
That said, I’ve never had good luck with REPLs, so I might be missing
something about that workflow.
[ppx_expect] <https://github.com/janestreet/ppx_expect>
Later in the thread, Simon Cruanes said
───────────────────────────────────────
Any test can be exploratory if they’re expect tests, imho
:slightly_smiling_face: . The core requirement is to have printers for
your types (e.g. `ppx_deriving.show'), so you can just create values
and use
┌────
│ Format.printf "my thing is now %a at ." pp_thing my_thing;;
└────
and see the output in dune. There’s a few annoyances (e.g. when it
crashes you need a bit of elbow grease to get the backtrace) but for
the most part it’s a nice workflow. I don’t apply it to everything,
though.
Kiran Gopinathan said
─────────────────────
W.r.t to development cycle in OCaml, I find myself working in 3
distinct ways depending on at which point of a project I am at:
• *small experiments* - sometimes I need a particular function that
isn’t provided by the stdlib, or some other bespoke well-defined
function. In these cases, I write the function directly in the
source code - using merlin’s typing information etc. to develop as
usual. Once I’ve done that, just to check the behaviour is as I
would expect, I copy over the definition to utop - if the function
depends on any larger state from my project, I write a dummy module
to mock these parts.
• *exploring a stateful system* - sometimes, I’m interacting with some
kind of stateful existing system that can be hard to run in utop
(for example, it might be a pain to get the library working in
bytecode) - e.g. gtk. In these cases, I setup a small executable
module that performs the minimum required to setup the initial
state, and do my iterative experiments by recompiling after each
change. Because the module is small, and the OCaml compiler/dune is
fast, this leads to a quick iterative editing experience.
• *working on a well-known codebase* - in all other cases, I’m working
on a codebase that I either know fairly well, or has sufficient
documentation and typing-discipline to make it easy to understand
what’s going on. In these cases, I can rely on the types and
documentation to make changes/add new functions, and if needed, use
tests to document expected behaviours.
As I’m speaking to an Emacs celebrity @bbatsov (thank you for your
great packages!), I’d be remiss if I didn’t mention my own little
experiment in this direction - `interactive-utop-mode':
<https://global.discourse-cdn.com/standard11/uploads/ocaml/original/2X/5/53856f555462c98314f99209cb3a145a02181428.webp>
<https://gitlab.com/-/snippets/2170216>
It’s not quite the same level of interactivity as lisp, granted, but
with judicious use of forking, you can emulate a slightly more
interactive repl experience.
Disclaimer: I don’t actually use this that much myself, it’s just more
of a proof of concept to show how it works.
Yaron Minsky said
─────────────────
Big +1 to expect tests for interactive programming. I wrote a bit
about this in an old blog post:
<https://blog.janestreet.com/repeatable-exploratory-programming/>
As to how to make things yet better:
On the ppx_expect, this could mean dropping dependencies.
At present, I think ppx_expect itself depends only on a couple other
ppx’s, and Base and Stdio, so I think it’s already reasonably light,
and the libraries in question are all portable.
From my perspective, a more important direction is to improve the
editor integration. Our internal expect test workflow is delightful,
and quite a bit better than what’s available publicly. When an error
pops up from the build, you hit a key to see the diff, hit another key
to accept the diff if you want to. It would be great if we could get
something similar to that in vscode.
Indeed, I posted an issue about this here:
<https://github.com/ocamllabs/vscode-ocaml-platform/issues/226>
Mirage retreat October 3rd - 9th
════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/mirage-retreat-october-3rd-9th/10363/1>
Hannes Mehnert announced
────────────────────────
Dear (aspiring) [MirageOS] hacker,
it is my please to announce that there will be the 11th MirageOS
retreat in early October in Mirleft, southern Morocco – yes, this time
at the seaside. Please find more details, including writeups of
earlier retreats at <http://retreat.mirage.io>
Everyone is welcome, be kind to each other. There won’t be much
Internet connectivity – but there’s plenty of beach, discussions,
impromptu talks, and a local network mainly constructed by MirageOS
unikernels.
If you have questions, don’t hesitate to ask them here in this thread,
or contact me directly via eMail “my first name” at mehnert.org.
[MirageOS] <https://mirage.io>
Experience report: making a JavaScript library from an OCaml library with js_of_ocaml
═════════════════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/experience-report-making-a-javascript-library-from-an-ocaml-library-with-js-of-ocaml/10380/1>
John Whitington announced
─────────────────────────
I have recently compiled our whole codebase with `js_of_ocaml',
producing a PDF processing library with can be used from JavaScript
programs on the server, or within the browser.
I started with no knowledge of JavaScript or its ecosystem, and the
`js_of_ocaml' examples don’t touch on this particular use case, so I
thought it would be useful to write up a quick experience report. I’m
still pretty naive on the topic, so do take it with a pinch of salt.
Corrections welcome.
Perhaps you could make your favourite OCaml library accessible from
JavaScript?
Source code: [Coherentpdf.js].
[Coherentpdf.js] <https://github.com/coherentgraphics/coherentpdf.js>
Existing Code
╌╌╌╌╌╌╌╌╌╌╌╌╌
For this project, I used the OCaml source code from the PDF library
CamlPDF, the command line PDF tools CPDF, and the C interface to CPDF,
CPDFLIB. CPDFLIB is an API allowing access to the PDF library from C
via the OCaml FFI (and, therefore, from .NET, Java, Python etc).
[CamlPDF source] | [CPDF source] | [Cpdflib source]
This code is almost all OCaml, with a small amount of C code for
cryptography and compression.
We will be recompiling all this code with `js_of_ocaml', to yield a
JavaScript library which can access all the functions in CPDFLIB.
Since CPDFLIB is a flat API with no direct access to OCaml data
structures, this should be easy to access from JavaScript. Building a
JavaScript library which had to directly manipulate OCaml data
structures would be more difficult.
[CamlPDF source] <https://github.com/johnwhitington/camlpdf>
[CPDF source] <https://github.com/johnwhitington/cpdf-source>
[Cpdflib source] <https://github.com/johnwhitington/cpdflib-source>
Compilation procedure
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Our build procedure will be simple. Copy all the OCaml and C source
files from the CamlPDF, CPDF and CPDFLIB into our build directory and
then:
• compile and link a bytecode executable with ocamlc using a makefile
• call `js_of_ocaml' to turn the resulting bytecode file into
JavaScript
(dune can also be used for both these steps)
Why the C files? Because we need them to link the bytecode executable:
the C code will be thrown away by `js_of_ocaml', but we must get the
bytecode linked.
`Js_of_ocaml' informs us that we have missing primitives: the C code
we will need to replace with JavaScript later, but the thing builds,
and a couple of megabytes of JavaScript is produced. It doesn’t do
anything yet, because we have no way of calling into it. But we can
load it as a module in node even at this stage.
If we miss out the cpdflib source files, though, and just include
camlpdf and cpdf, we get the cpdf command line tools, which we can run
in node as a command line tool, and they work just fine (if we don’t
use compressed or encrypted files, of course):
┌────
│ $node cpdf.js -pages cpdfjsmanual.pdf
│ 152
└────
Magic! About five or six times slower than the native code version,
but it works with minimal effort. `Js_of_ocaml' has supplied an
alternative set of primitives for input and ouput, and an alternative
runtime, and they replaces OCaml’s transparently.
Replacing C with JavaScript
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Our C code will be thrown away by JavaScript, so we need to provide
alternatives. `js_of_ocaml' allows us to write them in JavaScript with
some special comments. It will then plug each in for its corresponding
OCaml external (C symbol in the linked OCaml bytecode). Here’s the
replacement for one of CamlZip’s functions, using node’s own zlib
library:
┌────
│ //Provides: camlpdf_caml_zlib_compress
│ //Requires: caml_bytes_of_array
│ //Requires: caml_array_of_bytes
│ function camlpdf_caml_zlib_compress(s)
│ {
│ var s2 = caml_array_of_bytes(s);
│ var buf = Buffer.from(s2);
│ var output = zlib.deflateSync(buf);
│ return caml_bytes_of_array(output);
│ }
└────
In this case, it was not a direct replacement - the API is different.
So we must modify our OCaml code to be compilable in both OCaml and
`js_of_ocaml', like this:
┌────
│ (* js_of_ocaml only *)
│ external camlpdf_caml_zlib_compress : string -> string = "camlpdf_caml_zlib_compress"
│ external camlpdf_caml_zlib_decompress : string -> string = "camlpdf_caml_zlib_decompress"
│
│ let is_js =
│ Sys.backend_type = Sys.Other "js_of_ocaml"
│
│ let encode_flate stream =
│ if is_js then
│ Pdfio.bytes_of_string (camlpdf_caml_zlib_compress (Pdfio.string_of_bytes stream))
│ else
│ flate_process (Pdfflate.compress ~level:!flate_level) stream
└────
We also need some fake stubs in our C code to make sure it still
compiles in the non-JavaScript case with these new externals:
┌────
│ // So that the code links ok when using js_of_ocaml
│ char* camlpdf_caml_zlib_decompress(char *s) { return s; }
│ char* camlpdf_caml_zlib_compress(char *s) { return s; }
└────
This code will never be called, of course: it is simply to make the
symbol available.
Writing the JavaScript interface
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Now, we use the provided PPX `js_of_ocaml-ppx' to provide a JavaScript
interface to each function, and export them as functions in our
JavaScript module:
┌────
│ Js.export_all
│ (object%js
│ method fromFile filename userpw =
│ checkerror (Cpdflib.fromFile (Js.to_string filename) (Js.to_string userpw))
│ ...hundreds more functions...
│ )
└────
Notice `Js.to_string' here. Conversions to and from JavaScript will be
required for many data types, for example strings, arrays, bigarrays
and so on. In our case, `Cpdflib.fromFile' returns a simple integer,
so no conversion is required.
Trying it out
╌╌╌╌╌╌╌╌╌╌╌╌╌
Now that we have the library working, we can try it out by running a
JavaScript file with node, or by typing into the node REPL:
┌────
│ //Load coherentpdf.js
│ const coherentpdf = require('./coherentpdf.js');
│
│ var pdf = coherentpdf.fromFile('hello.pdf', '');
│ var merged = coherentpdf.mergeSimple([pdf, pdf, pdf]);
│ coherentpdf.toFile(merged, 'merged.pdf', false, false);
│
│ coherentpdf.deletePdf(pdf);
│ coherentpdf.deletePdf(merged);
└────
Compiling for the browser
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
We can use the [browserify] tool to bundle up our code and its
external libraries, including the parts of the node standard library
we use, to make single JavaScript file for use within a web page. This
nearly doubles the size:
┌────
│ 2192588 coherentpdf.browser.js
│ 1136687 coherentpdf.js
└────
Here is an example of a web page which uses coherentpdf.js to process
a PDF file entirely in the browser. You can choose a file, and it will
be processed and a download initiated with the output:
[https://coherentpdf.com/coherentpdfjs/index.html]
Of course, we have a problem now: in the browser, there are no files
to read to or write from - the browser is a sandboxed environment. So
we must make sure our API also contains functions to read to and write
from PDF files represented as byte arrays.
There is an additional issue: JavaScript in the browser does not cope
well with large chunks of synchronous code like ours: processing a big
PDF file would lock up the browser (or at least the tab). We must use
what is called a “web worker” to run it in the background in another
JavaScript environment, and communicate by message-passing only. See
the file `cpdfworker.js' for this code.
[browserify] <https://browserify.org>
[https://coherentpdf.com/coherentpdfjs/index.html]
<https://coherentpdf.com/coherentpdfjs/index.html>
Minification
╌╌╌╌╌╌╌╌╌╌╌╌
The [uglify-js] tool can be used to minify the JavaScript for
deployment on the web:
┌────
│ 2192588 coherentpdf.browser.js
│ 1324660 coherentpdf.browser.min.js
│ 1136687 coherentpdf.js
│ 849949 coherentpdf.min.js
└────
We are now down to 1.3Mb. Many web servers can serve gzip’d content
too, so that helps further, and we get down to 514Kb actually sent
over the web.
[uglify-js] <https://www.npmjs.com/package/uglify-js>
Documentation
╌╌╌╌╌╌╌╌╌╌╌╌╌
Because we built our library from within OCaml, and had it generated
for us by `js_of_ocaml-ppx', there is no place to put the docstrings.
So, unfortunately, we must write a separate JavaScript source file
with empty functions like this:
┌────
│ /** Returns the number of pages in a given PDF, with given user password. It
│ tries to do this as fast as possible, without loading the whole file.
│ @arg {string} password user password
│ @arg {Uint8Array} data PDF file as a byte array
│ @return {number} number of pages */
│ function pagesFastMemory(password, data) {}
└────
Now we can run any standard JavaScript documentation generator over
this to produce the HTML documentation.
Publishing the package
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Publishing the package on the npm package system is alarmingly easy -
a `package.json' file, mostly autogenerated, is added, and then it is
published from the command line. You can see it here:
[https://www.npmjs.com/package/coherentpdf]
Here is the source, including the `package.json': [Coherentpdf.js].
[https://www.npmjs.com/package/coherentpdf]
<https://www.npmjs.com/package/coherentpdf>
[Coherentpdf.js] <https://github.com/coherentgraphics/coherentpdf.js>
Licensing
╌╌╌╌╌╌╌╌╌
Our command line PDF tools and C/.NET/Java/Python APIs are under a
commercial license, or alternatively a non-standard “not for
commercial use” license. This is tiresome, because this non-standard
license prevents them being included in, for example, linux
distributions.
For coherentpdf.js, the license is just for the JavaScript output of
`js_of_ocaml', not the original OCaml source, so it can be given a
more standard AGPL license, whilst still being available for purchase,
and without opening up the commercial OCaml code. The AGPL isn’t
everyone’s favourite license, of course, but we start there for now.
To do
╌╌╌╌╌
What doesn’t work yet? Just two things:
• The default stack available in many browsers is small (and some have
a smaller stack for web workers), so coherentpdf.js can choke on
very large PDF files. This is not a problem in node on the server.
This will have to be fixed by modifications to the OCaml code
itself.
• `js_of_ocaml' installs its own top-level error handler, with the
unfortunate side-effect that any error in the node REPL after the
module is loaded - even a syntax error - causes the REPL to exit. I
plan to produce a patch to make this behaviour optional.
Final Remarks
╌╌╌╌╌╌╌╌╌╌╌╌╌
`js_of_ocaml' is a remarkably solid piece of kit. To be able to
recompile fifteen years of accumulated code with just a few changes
was very surprising to me.
Thanks to the `js_of_ocaml' team and others for answering all my
questions during this process, and correcting some of my
misconceptions.
I’m still very much a JavaScript newbie, and it’s not a language or
platform I’ve grown to love, but if you want your OCaml code to run in
the browser, or your OCaml library to be available to millions of
JavaScript programmers, you might try giving it a go.
Will I ever understand Format? or How to print a list vertically with indentation
═════════════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/will-i-ever-understand-format-or-how-to-print-a-list-vertically-with-indentation/10289/16>
Deep in this thread, Gaga said
──────────────────────────────
There is also a nice document called “Format Unraveled”, if you want
to learn more about `Format'. It can be found [here]
[here]
<https://hal.archives-ouvertes.fr/hal-01503081/file/format-unraveled.pdf>
GNU Guile 1.0.0 - Bindings to GNU Guile for OCaml
═════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-gnu-guile-1-0-0-bindings-to-gnu-guile-for-ocaml/10393/1>
Kiran Gopinathan announced
──────────────────────────
Hi everyone! I am pleased to announce a brand new package for the
OCaml ecosystem: Guile-ocaml
Guile-ocaml is a Free Software library that provides
high-level OCaml bindings to the FFI interface for GNU
Guile Scheme. The aim of these bindings are to provide an
easy way for OCaml developers to extend their OCaml
applications with GNU Guile scheme scripting capabilities,
providing simple combinators to translate terms and send
queries between the two languages.
You can find the documentation [here] - alongside a fairly
comprehensive documentation of all the bindings, it provides a step by
step guide on implementing the classical GNU Guile based turtle
drawing program, except using OCaml’s graphics module.
Here’s a sneak preview from that guide of what passing an OCaml
function to GNU Guile looks like using these bindings:
┌────
│ let move_by n =
│ if not @@ Guile.Number.is_integer n then
│ failwith "expected numeric arg";
│ let n = Guile.Number.int_from_raw n in
│ let x, y =
│ let cur_pos = Graphics.current_point () in
│ move n cur_pos !direction in
│ if !pen_down then
│ Graphics.lineto x y;
│ Graphics.moveto x y;
│ Guile.eol
│
│ let () = ignore @@ Guile.Functions.register_fun1 "move-by" move_by
└────
Also, I’d like to give extra thanks to opam-repository’s tireless
maintainers! The package was in limbo for a while because I couldn’t
work out why conf-guile was failing to build on certain distributions,
but thankfully @mseri and @kit-ty-kate were able to fix the issue!
[here] <https://gopiandcode.github.io/guile-ocaml/>
Update on Eio (effects-based direct-style IO for OCaml 5)
═════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/update-on-eio-effects-based-direct-style-io-for-ocaml-5/10395/1>
Thomas Leonard announced
────────────────────────
[Eio] provides an effects-based direct-style IO stack for OCaml 5.0.
It aims to be easy to use, secure, well documented, and fast. It
consists of a generic cross-platform API, plus optimised backends for
different platforms. It can be used instead of (or alongside) Lwt and
Async.
[Eio] <https://github.com/ocaml-multicore/eio>
Recent changes
╌╌╌╌╌╌╌╌╌╌╌╌╌╌
There have been several releases since the last announcement here. The
bigger changes include:
• [Fiber-local storage] (thanks to Jonathan Coates). This allows e.g.
a web-server to attach a request ID to the fiber that is handling
it, and then display this in all log messages generated by that
request.
• [Eio.Mutex] and [Eio.Condition] (@Lortex). These are similar to the
ones in the standard library, but they allow other fibers to run
while waiting instead of blocking the whole domain.
• [Eio.Buf_write] is a port of Faraday to Eio, allowing efficient
output buffering.
• [Fiber.fork_daemon] allows spawning a fiber that will be cancelled
automatically when the non-daemon fibers have finished. This is
useful for e.g. running a thread to collect entropy for a random
number generator while a program is running, without it preventing
the program from finishing.
• [Fiber.{iter,map,filter,fiter_map}] provide concurrent versions of
the corresponding operations in `List'.
• [Eio.Path] provides file-system access. New operations include
`read_dir' (@patricoferris), `unlink', `rmdir' and `rename'.
• [The networking APIs] now include UDP (@patricoferris) and DNS
(@bikalgurung), and IPv6 now works (@haesbaert).
• [Eio_mock] provides a framework for creating mocks for testing,
along with pre-defined ones for flows and networks.
`Eio_mock.Backend' is a special backend for tests. It does no real
IO, but can report if your tests deadlock.
• The much-requested [Eio_unix.sleep] is now available as a direct
replacement for `Lwt_unix.sleep'.
See the [release notes] for full details, and [the tutorial] for an
introduction.
[Fiber-local storage]
<https://ocaml-multicore.github.io/eio/eio/Eio/Fiber/index.html#fiber-local-variables>
[Eio.Mutex]
<https://ocaml-multicore.github.io/eio/eio/Eio/Mutex/index.html>
[Eio.Condition]
<https://ocaml-multicore.github.io/eio/eio/Eio/Condition/index.html>
[Eio.Buf_write]
<https://ocaml-multicore.github.io/eio/eio/Eio/Buf_write/index.html>
[Fiber.fork_daemon]
<https://ocaml-multicore.github.io/eio/eio/Eio/Fiber/index.html#val-fork_daemon>
[Fiber.{iter,map,filter,fiter_map}]
<https://ocaml-multicore.github.io/eio/eio/Eio/Fiber/index.html#concurrent-list-operations>
[Eio.Path]
<https://ocaml-multicore.github.io/eio/eio/Eio/Path/index.html>
[The networking APIs]
<https://ocaml-multicore.github.io/eio/eio/Eio/Net/index.html>
[Eio_mock]
<https://ocaml-multicore.github.io/eio/eio/Eio_mock/index.html>
[Eio_unix.sleep]
<https://ocaml-multicore.github.io/eio/eio/Eio_unix/index.html#val-sleep>
[release notes] <https://github.com/ocaml-multicore/eio/releases>
[the tutorial] <https://github.com/ocaml-multicore/eio#getting-ocaml-50>
Integration with Lwt and Async
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
[Lwt_eio] allows running Lwt and Eio code together in a single domain.
The 0.2 release adds support for integrating Lwt and Eio cancellation.
The [porting guide] shows how to update an Lwt application to Eio
incrementally.
[Async_eio] allows running Async and Eio code together in a single
domain. This is experimental and requires some changes to Async
itself.
[async-eio-lwt-chimera] shows how Async, Eio and Lwt code can all be
used together in a single event loop. It runs an Async server that
handles connections using an Lwt handler that reads a line from the
request and then handles it by using Eio to read the named file from
its data directory.
[Lwt_eio] <https://github.com/ocaml-multicore/lwt_eio>
[porting guide]
<https://github.com/ocaml-multicore/lwt_eio#porting-a-lwt-application-to-eio>
[Async_eio] <https://github.com/talex5/async_eio>
[async-eio-lwt-chimera]
<https://github.com/talex5/async-eio-lwt-chimera>
Porting
╌╌╌╌╌╌╌
It’s useful for people to try porting applications and libraries to
Eio so we can get feedback on the APIs and prioritise missing
features. The Lwt and Async porting guides linked above may be
helpful. Some examples so far include:
• [ocaml-cohttp] (@bikalgurung)
• [ocaml-geojson] (@patricoferris)
• [capnp-rpc] (@talex5)
• [rawlink] (@haesbaert)
• [dream] (@talex5)
• [mirage] (@Lortex)
• [mirage-crypto] (@BikalGurung)
[ocaml-cohttp]
<https://github.com/mirage/ocaml-cohttp/tree/master/cohttp-eio>
[ocaml-geojson] <https://github.com/geocaml/ocaml-geojson/tree/effects>
[capnp-rpc] <https://github.com/mirage/capnp-rpc/pull/256>
[rawlink] <https://github.com/haesbaert/rawlink>
[dream] <https://github.com/aantron/dream/pull/194>
[mirage] <https://github.com/TheLortex/mirage-monorepo>
[mirage-crypto] <https://github.com/mirage/mirage-crypto/pull/155>
Backends
╌╌╌╌╌╌╌╌
The recent [uring releases] have brought improved performance and
several new features to the `eio_linux' backend.
We are still hoping to persuade a Windows user to try using Eio. The
Luv backend should mostly work, but there are likely some problems
with paths, etc. Even better would be if someone writes a dedicated
`eio_windows' backend.
To add a backend for a new platform, it is best to start by studying
[Eio_mock.Backend], which is very simple as it does no IO. To add IO,
you can copy the pattern in either `eio_linux' (which provides its own
event loop that asks the OS to sleep) or the one in `eio_luv' (which
delegates to an event loop provided by another library).
[uring releases]
<https://github.com/ocaml-multicore/ocaml-uring/releases>
[Eio_mock.Backend]
<https://github.com/ocaml-multicore/eio/blob/main/lib_eio/mock/backend.ml>
OCaml at First Glance
═════════════════════
Archive: <https://discuss.ocaml.org/t/ocaml-at-first-glance/10396/1>
Bozhidar Batsov announced
─────────────────────────
This morning I spent a bit of time writing down some of [my initial
observations, experiences and thoughts about OCaml]. Hopefully they’ll
be useful to other newcomers to the language.
TLDR; I can’t say that my initial experience with OCaml was
mind-blowing, but it was definitely pleasant and with time the
language grows on me. I miss the simplicity and uniformity of Clojure
(still my favorite programming language by far) and Lisps in general
or some of Haskell’s goodies (e.g. typeclasses, [Hackage] and `ghci'),
but I feel OCaml strikes a very good balance between functional
programming, pragmatism and performance. It’s a `fun' language to work
with!
I’d be really curious to hear your own thoughts on the subject and to
receive feedback about any mistakes I might have made due to not being
familiar enough with OCaml and its ecosystem.
[my initial observations, experiences and thoughts about OCaml]
<https://batsov.com/articles/2022/08/29/ocaml-at-first-glance/>
[Hackage] <https://hackage.haskell.org/>
Kiran Gopinathan replied
────────────────────────
Nice blog post!
I’d say the development tooling for OCaml is pretty
decent, although I wouldn’t go as far as saying it’s
great.
Finally, a shameless plug, if you’re more familiar with lisp
development tooling, have you tried [gopcaml-mode]? It provides
paredit style structural editing support for OCaml:
<https://global.discourse-cdn.com/standard11/uploads/ocaml/original/2X/9/99180dcb28ea3021307e66801d8ee6f9f9b2c1b5.gif>
[gopcaml-mode] <https://github.com/gopiandcode/gopcaml-mode>
GADTs for typed option getter/setter
════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/gadts-for-typed-option-getter-setter/10398/1>
Romain Beauxis announced
────────────────────────
I’ve had a nice use-case of GADTs recently in [ocaml-srt] that I
thought I would share.
On the `C' side, the library implements an API /a la/ `syscall' with
named options and polymorphic types:
┌────
│ SRT_API int srt_getsockopt (SRTSOCKET u, int level /*ignored*/, SRT_SOCKOPT optname, void* optval, int*
│ optlen);
│ SRT_API int srt_setsockopt (SRTSOCKET u, int level /*ignored*/, SRT_SOCKOPT optname, const void* optval, int
│ optlen);
└────
And the socket options are listed in an `enum':
┌────
│ typedef enum SRT_SOCKOPT {
│
│ SRTO_MSS = 0, // the Maximum Transfer Unit
│ SRTO_SNDSYN = 1, // if sending is blocking
│ SRTO_RCVSYN = 2, // if receiving is blocking
│ SRTO_ISN = 3, // Initial Sequence Number (valid only after srt_connect or srt_accept-ed sockets)
│ SRTO_FC = 4, // Flight flag size (window size)
│ SRTO_SNDBUF = 5, // maximum buffer in sending queue
│ SRTO_RCVBUF = 6, // UDT receiving buffer size
│ ...
└────
On the OCaml side, it is then pretty natural to also want a
polymorphic API;
┌────
│ type 'a socket_opt
│
│ val messageapi : bool socket_opt
│ val payloadsize : int socket_opt
│ val rcvsyn : bool socket_opt
│ val sndsyn : bool socket_opt
│
│ val getsockflag : socket -> 'a socket_opt -> 'a
│ val setsockflag : socket -> 'a socket_opt -> 'a -> unit
└────
This can be achieved in a cool type-safe way and without writing any C
by combining [ctypes] and GADTs!
First, we export the socket option enums as OCaml polymorphic variants
using the `ctypes' API:
┌────
│ type socket_opt =
│ [ `Messageapi
│ | `Payloadsize
│ | `Rcvsyn
│ | `Sndsyn]
│
│ let socket_opt : socket_opt typ =
│ enum "SRT_SOCKOPT"
│ [
│ (`Messageapi, constant "SRTO_MESSAGEAPI" int64_t);
│ (`Payloadsize, constant "SRTO_PAYLOADSIZE" int64_t);
│ (`Rcvsyn, constant "SRTO_RCVSYN" int64_t);
│ (`Sndsyn, constant "SRTO_SNDSYN" int64_t)
│ ]
└────
Next, on our public OCaml API side, we use regular variant types with
a type annotation to use for GADTs pattern matching and match then
with the polymorphic variants returned by `ctypes':
┌────
│ type _ socket_opt =
│ | Messageapi : bool socket_opt
│ | Payloadsize : int socket_opt
│ | Rcvsyn : bool socket_opt
│ | Sndsyn : bool socket_opt
│
│ let messageapi = Messageapi
│ let payloadsize = Payloadsize
│ let rcvsyn = Rcvsyn
│ let sndsyn = Sndsyn
│
│ let srt_socket_opt_of_socket_opt (type a) : a socket_opt -> Srt.socket_opt =
│ function
│ | Messageapi -> `Messageapi
│ | Payloadsize -> `Payloadsize
│ | Rcvsyn -> `Rcvsyn
│ | Sndsyn -> `Sndsyn
└────
`ctypes' also. gives us bindings to the `get~/~set' functions from the
`C' library:
┌────
│ let setsockflag =
│ foreign "srt_setsockflag"
│ (int @-> socket_opt @-> ptr void @-> int @-> returning int)
│
│ let getsockflag =
│ foreign "srt_getsockflag"
│ (int @-> socket_opt @-> ptr void @-> ptr int @-> returning int)
└────
Now, we’re ready to implement our `get~/~set' polymorphic functions:
┌────
│ let getsockflag : type a. socket -> a socket_opt -> a =
│ fun sock opt ->
│ let arg = allocate int 0 in
│ let arglen = allocate int (sizeof int) in
│ ignore
│ (check_err
│ (getsockflag sock
│ (srt_socket_opt_of_socket_opt opt)
│ (to_voidp arg) arglen));
│ let to_bool () = !@arg <> 0 in
│ let to_int () = !@arg in
│ match opt with
│ | Rcvsyn -> to_bool ()
│ | Sndsyn -> to_bool ()
│ | Messageapi -> to_bool ()
│ | Payloadsize -> to_int ()
│
│ let setsockflag : type a. socket -> a socket_opt -> a -> unit =
│ fun sock opt v ->
│ let f t v = to_voidp (allocate t v) in
│ let of_bool v =
│ let v = if v then 1 else 0 in
│ (f int v, sizeof int)
│ in
│ let of_int v = (f int v, sizeof int) in
│ let arg, arglen =
│ match opt with
│ | Rcvsyn -> of_bool v
│ | Sndsyn -> of_bool v
│ | Messageapi -> of_bool v
│ | Payloadsize -> of_int v
└────
*⚠️ Question ⚠️*
I wasn’t able to join cases of the same type, for instance this:
┌────
│ match opt with
│ | Rcvsyn
│ | Sndsyn
│ | Messageapi -> to_bool ()
│ | Payloadsize -> to_int ()
└────
Is this a theoretical or implementation limitation?
[ocaml-srt] <https://github.com/savonet/ocaml-srt>
[ctypes] <https://github.com/ocamllabs/ocaml-ctypes>
yallop replied
──────────────
It’s an implementation limitation. There are some details at
<https://github.com/ocaml/ocaml/issues/5736>
Christian Lindig said
─────────────────────
Slightly related to the problem of sharing constants between the C
side and the OCaml side but much less sophisticated:
For my small epoll library [polly] I am creating a function in C using
a macro that returns the constant:
┌────
│ /* Make all constants available to clients by exporting them via a
│ * function. This avoids having to hard code them on the client side.
│ * */
│
│ #define CONSTANT(name) \
│ CAMLprim value caml_polly_ ## name(value unit) { return Val_int(name); }
│
│ CONSTANT(EPOLLIN);
│ CONSTANT(EPOLLPRI);
│ CONSTANT(EPOLLOUT);
└────
In the bindings I have one function per constant and call it once to
obtain the constant; the actual value of the constant is not part of
the OCaml code.
┌────
│ type t = int
│
│ external polly_IN : unit -> t = "caml_polly_EPOLLIN"
│ external polly_PRI : unit -> t = "caml_polly_EPOLLPRI"
│ external polly_OUT : unit -> t = "caml_polly_EPOLLOUT"
│
│ let inp = polly_IN ()
│ let pri = polly_PRI ()
│ let out = polly_OUT ()
└────
[polly] <https://github.com/lindig/polly>
shuttle v0.3.1 released
═══════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-shuttle-v0-3-1-released/8684/2>
Anurag Soni announced
─────────────────────
There have been some significant changes since this last release
(Versions 0.4.0 and 0.5.0 are available on opam):
• Buffered reader has a new utility method that allows reading lines
• Shuttle now supports file descriptors that don’t support nonblocking
I/O. For blocking I/O Shuttle uses async’s support for running
syscalls in its thread pool
• Buffered reader’s api has been simplified to remove
`read_one_chunk_at_a_time' in favor of a more familiar `read',
`read_line' etc. `refill' operation is supported to perform a read
syscall to fill a channel’s buffer, and `Input_channel.view' can be
used to get a view into the channel’s underlying buffer.
• Supports v0.15 series of the janestreet libraries
• Buffered reader now uses an auto-growing buffer instead of a fixed
size buffer than notified users that the internal buffer is full and
no progress can be made unless some content is consumed. This should
allow starting with a smaller buffer without needing to worry about
implementing some client side buffering to hold unconsumed data.
Channels allow configuring an upper bound on the internal buffer
length, if a buffer grows beyond that an exception is raised.
• Buffered writer’s support a richer flush interface. Flush operations
report errors encountered while attempting to write any pending
bytes. This results in a flush operation that returns a deferred
that will resolve at some point with either a success or an error,
instead of the older flush operation that would return a deferred
that never resolved if there was an error during a write syscall.
coinductive data types
══════════════════════
Archive:
<https://sympa.inria.fr/sympa/arc/caml-list/2022-08/msg00009.html>
Aaron Gray asked and François Pottier replied
─────────────────────────────────────────────
Does either ML or OCaML have coinductive data types ? And
if so could you please point me at the/some appropriate
documentation on them.
ML and OCaml have algebraic data types, which are recursive (that is,
more general than inductive and co-inductive types). Algebraic data
type definitions are not subject to a positivity restriction, and
algebraic data types can be constructed and deconstructed by recursive
functions, which are not subject to a termination check.
If you want to see a typical example of a “co-inductive” data
structure encoded in OCaml, I suggest to have a look at “sequences”,
which can be described as potentially infinite lists:
<https://v2.ocaml.org/api/Seq.html>
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I’ll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <https://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <https://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20220830/97d3cb2a/attachment-0001.html>
More information about the caml-news-weekly
mailing list