[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Sep 14 05:06:32 PDT 2021
Hello
Here is the latest OCaml Weekly News, for the week of September 07 to
14, 2021.
Table of Contents
─────────────────
opam-grep: search through the sources of all the packages in opam-repository
The OCaml 4.13 preview for Merlin is now available
Outreachy Summer 2021
http-cookie v4.2.0
Dune 2.9.1
MDX 1.11.0
OCaml 4.13.0, first release candidate
New release: ProVerif 2.03
ocaml-lsp-server 1.8.0
Multicore OCaml: August 2021
Higher-kinded bounded polymorphism
OCaml Workshop 2021: Live Stream
Windows-friendly OCaml 4.12 distribution 2nd preview release (0.2.0)
Old CWN
opam-grep: search through the sources of all the packages in opam-repository
════════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-opam-grep-search-through-the-sources-of-all-the-packages-in-opam-repository/8434/1>
Kate announced
──────────────
[opam-grep] is an opam plugin that greps through the sources of all
opam packages.
To install it, simply call:
┌────
│ $ opam install opam-grep
└────
Then to use it, simply call:
┌────
│ $ opam grep "your regexp"
└────
/Side note/: currently opam-grep will cache the sources in your cache
directory (`~/.cache/opam-grep'), so a few GB of available disk space
is most likely required.
I hope this can help core compiler and community library devs alike,
to know which part of their software is used in the wild.
[opam-grep] <https://github.com/kit-ty-kate/opam-grep>
The OCaml 4.13 preview for Merlin is now available
══════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-the-ocaml-4-13-preview-for-merlin-is-now-available/8436/1>
Kate announced
──────────────
Over the past previous OCaml releases the release-readiness team tried
its best to make most packages, and especially core OCaml packages
(dune, merlin, ppxlib, …) available day one so people can have the
smoothest transition possible to the new compiler release.
Today I'm announcing that merlin (arguably the trickiest core tool to
port to new releases, and the most indispensable) is now available to
install for early adopters of OCaml 4.13.
To install the latest OCaml 4.13 beta together with Merlin simply
call:
┌────
│ $ opam switch create 4.13 --formula='"ocaml-base-compiler" {>= "4.13" & < "4.14"}'
│ $ opam install merlin
└────
The package is still an early preview. If you encounter any problems
while using merlin, please feel free to report it directly in
<https://github.com/ocaml/merlin/pull/1381>
All of the [OCaml core packages] (with the exception of ocaml-lsp,
hopefully [soon]) should be available out-of-the-box. If some packages
aren't available yet, opam-alpha-repository should have you covered:
┌────
│ $ opam remote add alpha git://github.com/kit-ty-kate/opam-alpha-repository.git
└────
If you have any questions or remarks feel free to ask them here.
[OCaml core packages]
<https://github.com/ocaml/opam-repository/issues/18791>
[soon] <https://github.com/ocaml/ocaml-lsp/issues/503>
Outreachy Summer 2021
═════════════════════
Archive: <https://discuss.ocaml.org/t/outreachy-summer-2021/8438/1>
Patrick Ferris announced
────────────────────────
This summer the OCaml community rebooted its participation in
Outreachy "…a diversity initiative that provides paid, remote
internships to people subject to systemic bias and impacted by
underrepresentation in the technical industry where they are living".
We had the pleasure of working with three talented interns over 13
weeks this summer on projects relating to v3.ocaml.org. Each intern
has summarised their experience in a blog post that we'd like to share
with you.
*A quick reminder, if you are interested in mentoring for the OCaml
community in the next round, please [read this post] and [apply with
your project]. The project submission deadline is Sept. 23, 2021 at
4pm UTC.*
[read this post]
<https://discuss.ocaml.org/t/become-an-outreachy-mentor-support-the-growth-and-diversity-of-the-ocaml-community/8213>
[apply with your project]
<https://www.outreachy.org/communities/cfp/ocaml/>
A Graphql Endpoint for Packages on OCaml.org
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
One of the centre-pieces for the ocaml.org revamp is [centralised docs
for every version of every opam package]. With all of this information
it would be useful to expose an API for exploring the packages. This
is exactly what Odinaka Joy worked on and is live now at
v3.ocaml.org/graphiql! For more information [have a read of the
accompanying blog post].
[centralised docs for every version of every opam package]
<https://v3.ocaml.org/packages>
[have a read of the accompanying blog post]
<https://dinakajoy-blog.vercel.app/blog/my-journey>
A Modern Front-end for OCaml.org
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Another goal of v3.ocaml.org was to modernise the design and front-end
of the site. This primarily meant [splitting the data out] and using
modern tools like [tailwind, react, nextjs and rescript] to build a
new front-end. Shreya Kumari Gupta has been driving this effort
forward by porting more content, pages and designs into v3 including
adding all of the academic institutions, industrial users and solving
complicated nextjs-rescript-leaflet bindings. To learn about this and
Shreya's overall experience [have a read of that blog post].
[splitting the data out] <https://github.com/ocaml/ood>
[tailwind, react, nextjs and rescript]
<https://github.com/ocaml/v3.ocaml.org>
[have a read of that blog post]
<https://shreyaswikriti693235797.wordpress.com/2021/09/04/my-internship-with-ocaml-outreachy-summary/>
Peer-to-peer, privacy-focused video streaming for OCaml.org
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Last and certainly not least we have OCaml.org's peer-to-peer
[peertube] instance where videos (such as the [2021 OCaml Workshop])
are hosted with no tracking software and backed-up for
posterity. Diksha Gupta worked on adding content, building tools to
use the peertube API to query about the videos which ultimately
creates [a usable index of the videos in ood]. For more on this
project, [look no further than the blog post].
It's been a pleasure working with all of the interns this summer and
we're excited to see this initiative grow.
[peertube] <https://joinpeertube.org/>
[2021 OCaml Workshop]
<https://watch.ocaml.org/video-channels/ocaml2021/videos>
[a usable index of the videos in ood]
<https://github.com/ocaml/ood/blob/main/data/watch.yml>
[look no further than the blog post]
<https://dikshagupta99.wordpress.com/2021/09/04/outreachy-my-experience-and-my-project/>
http-cookie v4.2.0
══════════════════
Archive: <https://discuss.ocaml.org/t/ann-http-cookie-v4-2-0/8441/1>
Bikal Lem announced
───────────────────
I am pleased to announce v4.* series release of `http-cookie'. The v4
release contains many new enhancements to make working with cookies in
OCaml web applications a little bit easier.
v4.*.* release series features the following additions and changes:
1. Add pretty printers - `pp, pp_date_time and pp_same_site'. These
can be used during debugging and in a OCaml toplevel session
2. Flatten out `Same_site' module
3. Add `of_cookie' to parse `Cookie' header.
4. Add `of_set_cookie' to parse `Set-Cookie' header. This can be used
by http client applications
5. Rename `of_cookie_header' to `of_cookie', `to_cookie_header_value'
to `to_cookie', `to_set_cookie_header_value' to `to_set_cookie' to
enhance API ergonomics
6. Add validation for cookie `domain' value which consists of - host
name, IPv4 and IPv6 - to conform to their respective RFC specs -
Domain name RFC(RFC 1034), RFC 1123 and draft IPv4 and IPv6 RFCs.
7. Improve documentation
• [Documentation]
• [Github repo]
[Documentation]
<https://lemaetech.co.uk/http-cookie/http-cookie/Http_cookie/index.html>
[Github repo] <https://github.com/lemaetech/http-cookie>
Dune 2.9.1
══════════
Archive: <https://discuss.ocaml.org/t/ann-dune-2-9-1/8442/1>
Emilio Jesús Gallego Arias announced
────────────────────────────────────
on behalf of the Dune team I'm pleased to announce the release of Dune
2.9.1.
This is a minor release, fixing an important problem with opam file
generation when `(lang dune 2.9)' was set, and a few other minor
fixes; please find the changelog below:
• Don't use `subst --root' in Opam files
(<https://github.com/ocaml/dune#4806>, @MisterDA)
• Fix compilation on Haiku (<https://github.com/ocaml/dune#4885>,
@Sylvain78)
• Allow depending on `ocamldoc' library when `ocamlfind' is not
installed. (<https://github.com/ocaml/dune#4811>, fixes
<https://github.com/ocaml/dune#4809>, @nojb)
• Fix `(enabled_if ...)' for installed libraries
(<https://github.com/ocaml/dune#4824>, fixes
<https://github.com/ocaml/dune#4821>, @dra27)
• Create more future-proof opam files using
`--promote-install-files=false'
(<https://github.com/ocaml/dune#4860>, @bobot)
MDX 1.11.0
══════════
Archive: <https://discuss.ocaml.org/t/ann-mdx-1-11-0/8457/1>
Marek Kubica announced
──────────────────────
On behalf of the MDX maintainers I'd like to announce the availability
of [MDX 1.11.0]. This is mostly a maintenance release but it prepares
for a 2.0 release soon. The most important change in this regard is
that it will output deprecation warnings when OCaml toplevel lines are
not `;;' terminated.
As an example this will trigger a deprecation warning to be printed on
stderr:
┌────
│ # List.map (fun x -> x + 1) [1; 2; 3]
│ - : int list = [2; 3; 4
└────
Note that this will only print a warning, it will still process these
lines correctly and output code just like before. To avoid the
warning, make sure to add terminating `;;':
┌────
│ # List.map (fun x -> x + 1) [1; 2; 3];;
│ - : int list = [2; 3; 4
└────
To ease migration, MDX 2.0 will change behaviour and insert `;;' into
the output so users will be able to use e.g. `dune promote' to get
their source files updated with `;;'. In MDX 3.0 reading toplevel
phrases without the trailing `;;' will be finally unsupported and the
deprecation warning will be removed.
Happy documenting!
[MDX 1.11.0] <https://opam.ocaml.org/packages/mdx/mdx.1.11.0/>
Changed
╌╌╌╌╌╌╌
• Use odoc-parser.0.9.0 ([#333], @Juloo)
[#333] <https://github.com/realworldocaml/mdx/pull/333>
Deprecated
╌╌╌╌╌╌╌╌╌╌
• Add a deprecation warning for toplevel blocks that are not
terminated with `;;' ([#342], @Leonidas)
[#342] <https://github.com/realworldocaml/mdx/pull/342>
Fixed
╌╌╌╌╌
• Fix accidental redirect of stderr to stdout ([#343], @Leonidas)
• Remove trailing whitespaces that were added to indent empty lines
([#341], @gpetiot)
[#343] <https://github.com/realworldocaml/mdx/pull/343>
[#341] <https://github.com/realworldocaml/mdx/pull/341>
OCaml 4.13.0, first release candidate
═════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-4-13-0-first-release-candidate/8461/1>
octachron announced
───────────────────
The release of OCaml 4.13.0 is imminent. As a last test that
everything is in order, we are publishing a first release candidate
for OCaml 4.13.0.
Compared to the last beta release, this release candidate includes one
small fix for flambda, another for the compiler-libs, one regression
fix when compiling C files with ocamlc or ocamlopt, and a handful of
configuration and build system improvements (in particular on
Windows). See the changelog below for more details.
If you find any bugs, please report them here:
<https://github.com/ocaml/ocaml/issues>
The release of OCaml 4.13.0 is currently planned for mid-September.
An assembler generation bug was discovered this morning, we might
release another release candidate next week once this bug is fixed.
Happy hacking, Florian Angeletti for the OCaml team.
Installation instructions
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
The base compiler can be installed as an opam switch with the
following commands
┌────
│ opam update
│ opam switch create 4.13.0~rc1 --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
└────
If you want to tweak the configuration of the compiler, you can switch
to the option variant with:
┌────
│ opam update
│ opam switch create <switch_name> --packages=ocaml-variants.4.13.0~rc1+options,<option_list>
│ --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
└────
where <option_list> is a comma separated list of ocaml-option-*
packages. For instance, for a flambda and no-flat-float-array switch:
┌────
│ opam switch create 4.13.0~rc1+flambda+nffa
│ --packages=ocaml-variants.4.13.0~rc1+options,ocaml-option-flambda,ocaml-option-no-flat-float-array
│ --repositories=default,beta=git+https://github.com/ocaml/ocaml-beta-repository.git
└────
All available options can be listed with "opam search ocaml-option".
The source code for the beta release is also available at these
addresses:
• <https://github.com/ocaml/ocaml/archive/4.13.0-rc1.tar.gz>
• <https://caml.inria.fr/pub/distrib/ocaml-4.13/ocaml-4.13.0~rc1.tar.gz>
Changes since the first beta
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Bug fixes
┄┄┄┄┄┄┄┄┄
• [#10593]: Fix untyping of patterns without named existential
quantifiers. This bug was only present in the beta version of OCaml
4.13.0. (Ulysse Gérard, review by Florian Angeletti)
• [#10603], [#10611]: Fix if condition marked as inconstant in flambda
(Vincent Laviron and Pierre Chambart, report by Marcello Seri)
[#10593] <https://github.com/ocaml/ocaml/issues/10593>
[#10603] <https://github.com/ocaml/ocaml/issues/10603>
[#10611] <https://github.com/ocaml/ocaml/issues/10611>
Regression fix
┄┄┄┄┄┄┄┄┄┄┄┄┄┄
• [#9960], [#10619](new in rc1): extend ocamlc/ocamlopt's -o option to
work when compiling C files (Sébastien Hinderer, reported by Daniel
Bünzli, review by Florian Angeletti and Gabriel Scherer)
[#9960] <https://github.com/ocaml/ocaml/issues/9960>
[#10619] <https://github.com/ocaml/ocaml/issues/10619>
Internal configuration and build system
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
• [#10471]: Fix detection of arm32 architectures with musl in
configure. (Louis Gesbert, review by David Allsopp)
• [#10451]: Replace the use of iconv with a C utility to convert
$(LIBDIR) to a C string constant on Windows when building the
runtime. Hardens the generation of the constant on Unix for paths
with backslashes, double-quotes and newlines. (David Allsopp, review
by Florian Angeletti and Sébastien Hinderer)
• [#10511]: Cygwin ports now correctly configure when flexdll is not
available. (David Allsopp, review by Florian Angeletti)
• [#10584], [#10856]: Standard Library documentation build no longer
fails if optional libraries have been disabled. (David Allsopp,
report by Yuri Victorovich review by Florian Angeletti)
[#10471] <https://github.com/ocaml/ocaml/issues/10471>
[#10451] <https://github.com/ocaml/ocaml/issues/10451>
[#10511] <https://github.com/ocaml/ocaml/issues/10511>
[#10584] <https://github.com/ocaml/ocaml/issues/10584>
[#10856] <https://github.com/ocaml/ocaml/issues/10856>
Manual
┄┄┄┄┄┄
• [#10605]: manual, name few css classes to ease styling and
maintainability. (Florian Angeletti, review by Wiktor Kuchta and
Gabriel Scherer)
[#10605] <https://github.com/ocaml/ocaml/issues/10605>
New release: ProVerif 2.03
══════════════════════════
Archive:
<https://discuss.ocaml.org/t/new-release-proverif-2-03/8463/1>
Bruno Blanchet announced
────────────────────────
Vincent and I are pleased to announce a new release of ProVerif,
version 2.03. ProVerif is a automatic security protocol verifier that
relies on the symbolic model of cryptography.
ProVerif is available at [http://proverif.inria.fr ] It is also
available via opam (see installation instructions in Section 1.4 of
the manual [http://proverif.inria.fr/manual.pdf ] )
This release contains many important changes since version 2.01
(version 2.02 was not announced here because of a pending anonymous
submission), mainly implemented by Vincent:
• Change of license for Windows binaries: Windows binaries are now
distributed under GPL, like the source (because ProVerif now
integrates a modified version of an OCaml module under LGPL). If
this change of license prevents you from using ProVerif, please
email me.
• Major speedups for large examples.
• Several extensions, including temporal variables in correspondence
queries (see Section 4.3.1 of the manual, paragraph "Temporal
correspondences").
• Bug fixes.
[http://proverif.inria.fr ] <http://proverif.inria.fr/>
[http://proverif.inria.fr/manual.pdf ]
<http://proverif.inria.fr/manual.pdf>
ocaml-lsp-server 1.8.0
══════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ocaml-lsp-server-1-8-0/8470/1>
Rudi Grinberg announced
───────────────────────
On behalf of the ocaml-lsp team, I'm pleased to announce version
1.8.0. This release contains some quality of life bug fixes, better
diagnostics locations, and a few new code actions. Happy hacking.
The full change log is rendered below for your convenience:
Features
╌╌╌╌╌╌╌╌
• Add a new code action `Add missing rec keyword', which is available
when adding a `rec' keyword can fix `Unbound value ...' error, e.g.,
┌────
│ let fact n = if n = 0 then 1 else n * fact (n - 1)
│ (* ^^^^ Unbound value fact *)
└────
Adding `rec' to the definition of `fact' will fix the problem. The
new code action offers adding `rec'.
• Use ocamlformat to properly format type snippets. This feature
requires the `ocamlformat-rpc' opam package to be installed. (#386)
• Add completion support for polymorphic variants, when it is possible
to pin down the precise type. Examples (`<|>' stands for the cursor)
when completion will work (#473)
Function application:
┌────
│ let foo (a: [`Alpha | `Beta]) = ()
│
│ foo `A<|>
└────
Type explicitly shown:
┌────
│ let a : [`Alpha | `Beta] = `B<|>
└────
Note: this is actually a bug fix, since we were ignoring the
backtick when constructing the prefix for completion.
• Parse merlin errors (best effort) into a more structured form. This
allows reporting all locations as "related information" (#475)
• Add support for Merlin `Construct' command as completion
suggestions, i.e., show complex expressions that could complete the
typed hole. (#472)
• Add a code action `Construct an expression' that is shown when the
cursor is at the end of the typed hole, i.e., `_|', where `|' is the
cursor. The code action simply triggers the client (currently only
VS Code is supported) to show completion suggestions. (#472)
• Change the formatting-on-save error notification to a warning
notification (#472)
• Code action to qualify ("put module name in identifiers") and
unqualify ("remove module name from identifiers") module names in
identifiers (#399)
Starting from:
┌────
│ open Unix
│
│ let times = Unix.times ()
│ let f x = x.Unix.tms_stime, x.Unix.tms_utime
└────
Calling "remove module name from identifiers" with the cursor on the
open statement will produce:
┌────
│ open Unix
│
│ let times = times ()
│ let f x = x.tms_stime, x.tms_utime
└────
Calling "put module name in identifiers" will restore:
┌────
│ open Unix
│
│ let times = Unix.times ()
│ let f x = x.Unix.tms_stime, x.Unix.tms_utime
└────
Fixes
╌╌╌╌╌
• Do not show "random" documentation on hover
• fixed by [merlin#1364]
• fixes duplicate:
• [ocaml-lsp#344]
• [vscode-ocaml-platform#111]
• Correctly rename a variable used as a named/optional argument (#478)
• When reporting an error at the beginning of the file, use the first
line not the second (#489)
[merlin#1364] <https://github.com/ocaml/merlin/pull/1364>
[ocaml-lsp#344] <https://github.com/ocaml/ocaml-lsp/issues/344>
[vscode-ocaml-platform#111]
<https://github.com/ocamllabs/vscode-ocaml-platform/issues/111>
sid asked and Rudi Grinberg replied
───────────────────────────────────
I have a technical question. I notice that the code base
uses dune fibers for concurrency. What benefits do fibers
have over, say, Lwt/Async?
I wanted to write about more that in more detail at some point. I can
give a quick preview here I guess. The gist of it is that fibers are
much more lightweight and provide simpler error handling
semantics. You do lose out on choice (`Deferred.choose' and friends)
though.
Like deferred/lwt, a fiber is also a monadic value that represents
asynchronous computation. In deferred/lwt, binding on the computation
waits for it to finish, and then stores the result in a
ref. Subsequent binds will reuse the saved value. A fiber stores
nothing and will always re-run the computation from scratch to produce
the value. For example, this will print `foo' twice.
┌────
│ let foo = Fiber.of_thunk (fun () -> Fiber.return (print_endline "foo"))
│
│ let bar =
│ let* () = foo in
│ let* () = bar in
│ Fiber.return ()
└────
To save the value computed, one must opt-in and create an Ivar
manually. What's the advantage of this approach? Well, bind becomes a
lot simpler (and cheaper). I'll even paste the implementation to prove
how simple it is:
┌────
│ let ( >>= ) t f k = t (fun x -> f x k)
└────
Lwt's and Async's bind are far more complex (and slow).
Moving on to error handling. Lwt has the disaster that is
`Lwt.async'. Enough said. There's `Lwt.dont_wait' now, but it's still
fishy to me that yet another error handling channel is needed in
addition to `Lwt.catch'. With Async, things are cleaner with monitors,
but IMO, a completely separate concept for error handling is overkill.
To explain how fiber does error handling, I'll first explain the other
(aside from sequencing with bind) composition primitive for fibers. A
fiber may spawn sub fibers that will be executed concurrently. The
simplest primitive to do is that is the fork:
┌────
│ (* the main concurrency primitive *)
│ val fork_and_join : (unit -> 'a Fiber.t) -> (unit -> 'b Fiber.t) -> ('a * 'b) Fiber.t
└────
Before a fiber can finish and produce a value, it must wait for all of
its sub fibers to terminate. If a fiber raises an exception instead of
producing a value, the exception will be passed to the fiber's error
handler. If there' no error handler set, the exception will be passed
to the parent's fiber error handler (if the error handler itself
raises, the error will always be passed to the parent's error
handler).
The rules above are exhaustive and leave very little room for
ambiguity. In practice, I find it very convenient to write robust
error handling code this way. For example with lsp, it's easy to
install a handler in one place for a single request and know that
errors will not leak outside of it. It is also easy to make sure that
no spawned computation is left "dangling" after a request is served.
There's also some practical benefits that made me choose Fiber:
• Depending on async/lwt would impose version constraints for end
users on these key libraries. In contrast, fiber is a single module
that is trivial to vendor and is invisible to users.
• Fiber contains no C and hence very portable. This can be handy if
one wants to get ocamllsp working with an experimental compiler for
example.
There's downsides as well of course. Fiber is quite minimal and
therefore comes without a scheduler. I managed to throw something
together by mostly copying dune's scheduler, but it's not something
I'd recommend to anybody. If fiber ever becomes public, we'd need
something like libuv to provide us with a serious and reusable
scheduler.
That's all for now. It would be much more interesting to write about
how dune itself uses fibers, but that is better written by other
people from the team.
Multicore OCaml: August 2021
════════════════════════════
Archive:
<https://discuss.ocaml.org/t/multicore-ocaml-august-2021/8484/1>
Anil Madhavapeddy announced
───────────────────────────
Multicore OCaml: August 2021
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Welcome to the August 2021 [Multicore OCaml] monthly report! The
following update and the [previous updates] have been compiled by me,
@ctk21, @kayceesrk and @shakthimaan. This month's update is a bit
quieter as August is also a period of downtime in Europe (and our crew
in India also took well-deserved time off), but we all participated in
the [online OCaml Workshop] which was held virtually this year, so
there are plenty of videos to watch!
The multicore effort is all on track for integration into OCaml 5.0
early next year, with the core team currently organising the
upstreaming code review strategy over the coming winter months.
Meanwhile, there are some blog posts and videos from the OCaml
Workshop which give more detailed updates on both domains-parallelism
and effects.
• *Adapting the OCaml ecosystem for Multicore OCaml*
• /This talk covers how our community can adapt to the forthcoming
OCaml 5.0 with parallelism./
• [Blog post 1], [Blog post 2]
• [Video]
• *Parafuzz coverage guided Property Fuzzing for Multicore OCaml
programs*
• /We develop ParaFuzz, an input and concurrency fuzzing tool for
Multicore OCaml programs. ParaFuzz builds on top of Crowbar which
combines AFL-based grey box fuzzing with QuickCheck and extends it
to handle parallelism./
• [Video]
• *Experiences with Effects*
• /The multicore branch of OCaml adds support for effect
handlers. In this talk, we report our experiences with effects,
both from converting existing code, and from writing new code./
• [Video]
As always, the Multicore OCaml updates are listed first, which are
then followed by the updates from the Ecosystem libraries and Sandmark
benchmarking. Finally, the ICFP relevant blog posts and videos from
the OCaml workshop sessions are provided for your reference.
[Multicore OCaml] <https://github.com/ocaml-multicore/ocaml-multicore>
[previous updates] <https://discuss.ocaml.org/tag/multicore-monthly>
[online OCaml Workshop]
<https://discuss.ocaml.org/t/ocaml-workshop-2021-live-stream/8366>
[Blog post 1] <http://segfault.systems/blog/2021/adapting-to-multicore/>
[Blog post 2]
<https://tarides.com/blog/2021-08-26-tarides-engineers-to-present-at-icfp-2021>
[Video]
<https://watch.ocaml.org/videos/watch/629b89a8-bbd5-490d-98b0-d0c740912b02>
[Video]
<https://watch.ocaml.org/videos/watch/c0d591e0-91c9-4eaa-a4d7-c4f514de0a57>
[Video]
<https://watch.ocaml.org/videos/watch/74ece0a8-380f-4e2a-bef5-c6bb9092be89>
Multicore OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
• [ocaml-multicore/ocaml-multicore#632] Str module multi domain safety
An issue on stdlib safety in the OCaml `Str' module to work
concurrently with Multicore OCaml.
• [ocaml-multicore/ocaml-multicore#633] Error building 4.12.0+domains
with no-flat-float-arrays
A linker error observed by `Adrián Montesinos González' (`debugnik')
when installing 4.12.0+domains `no-flat-float-arrays'.
• [ocaml-multicore/ocaml-multicore#634] Strange type errors from
merlin (This expression has type string/1)
Type errors reported from merlin (4.3.1-412) when using the effects
version of the Multicore OCaml compiler.
• [ocaml-multicore/ocaml-multicore#624] core v0.14: test triggers a
segfault in the GC
The root cause of the segfault when running `core.v0.14' test suite
with Multicore OCaml 4.12.0+domains has been identified.
• [ocaml-multicore/ocaml-multicore#573] Backport trunk safepoints PR
to multicore
This is an on-going effort to backport the Safepoints implementation
to Multicore OCaml.
[ocaml-multicore/ocaml-multicore#632]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/632>
[ocaml-multicore/ocaml-multicore#633]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/633>
[ocaml-multicore/ocaml-multicore#634]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/634>
[ocaml-multicore/ocaml-multicore#624]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/624>
[ocaml-multicore/ocaml-multicore#573]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/573>
Ecosystem
╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#10] Edited
for flow/syntax/consistency
The Parallel Programming in Multicore OCaml chapter has been updated
for consistency, syntax flow and grammar.
• [ocaml-multicore/retro-httpaf-bench#15] Optimise Go code
The `nethttp-go/httpserv.go' benchmark has been optimised with use
of `Write' instead of `fmt.Fprintf', and the removal of yield().
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/4/480cac4d79f8d84b8fb597885cad8d69c8d0929e.png>
• [ocaml-multicore/ocaml-uring#37] `poll_add' test hangs on s390x
The use of `poll_add' causes a hang on `s390x' architecture. A
backtrace with GDB is provided for reference:
┌────
│ (gdb) bt
│ #0 0x000003ffb63ec01e in __GI___libc_write (nbytes=<optimized out>, buf=<optimized out>, fd=<optimized out>)
│ at ../sysdeps/unix/sysv/linux/write.c:26
│ #1 __GI___libc_write (fd=<optimized out>, buf=0x3ffffdee8e0, nbytes=1) at ../sysdeps/unix/sysv/linux/write.c:24
│ #2 0x000002aa0dbb0ca2 in unix_write (fd=<optimized out>, buf=<optimized out>, vofs=<optimized out>,
│ vlen=<optimized out>) at write.c:44
│ #3 0x000002aa0dbd4d3a in caml_c_call ()
└────
• [ocaml-multicore/domainslib#37] parallel_map
@UnixJunkie has provided a simplified version for the interface for
scientific parallel programming as recommended by the parany
library.
┌────
│ val run:
│ ?csize:int ->
│ ~nprocs: int ->
│ demux:(unit -> 'a) ->
│ work:('a -> 'b) ->
│ mux:('b -> unit) -> unit
└────
• [ocaml-multicore/domainslib#39] Add a fast path in parallel scan
A patch that performs a sequential scan when the number of elements
is less than or equal to the pool size or if the number of domains
is one.
• [ocaml-multicore/domainslib#40] Parallel map
A PR that implements `parallel_map' in lib/task.ml that includes an
optional chunk size parameter.
[ocaml-multicore/parallel-programming-in-multicore-ocaml#10]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/10>
[ocaml-multicore/retro-httpaf-bench#15]
<https://github.com/ocaml-multicore/retro-httpaf-bench/pull/15>
[ocaml-multicore/ocaml-uring#37]
<https://github.com/ocaml-multicore/ocaml-uring/issues/37>
[ocaml-multicore/domainslib#37]
<https://github.com/ocaml-multicore/domainslib/issues/37>
[ocaml-multicore/domainslib#39]
<https://github.com/ocaml-multicore/domainslib/pull/39>
[ocaml-multicore/domainslib#40]
<https://github.com/ocaml-multicore/domainslib/pull/40>
Completed
┄┄┄┄┄┄┄┄┄
• [ocaml-multicore/retro-httpaf-bench#13] Update EIO for performance
improvements, multiple domains
`httpf-eio' has been enhanced with performance improvements when
running with multiple domains. The results on an 8-core VM with 100
connections and 5 second runs is shown below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/4/44187373b7e054af11e2a6fe9b177545fc0d79f2.png>
The following illustration is from a VM for 1000 connections and 60
second runs:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/2/2866c07ef5956bc5451abb214d0947103e2fb268.png>
The results with `GOMAXPROCS=3' for three OCaml domains is as
follows:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/9/963e6df72ed1e12ff16f0f3a1a34c294ca11ddd4.png>
• [ocaml-multicore/ocaml-uring#36] Update to cstruct 6.0.1
`ocaml-uring' now uses `Cstruct.shiftv' and has been updated to use
cstruct.6.0.1.
• [ocaml-multicore/domainslib#41] Use the master branch in the link to
usage examples
The README.md file has been updated to point to the sample programs
in the master branch that use the new `num_additional_domains'
argument label.
• The Multicore OCaml concurrency bug detection tool named [ParaFuzz]
is now available in GitHub as Free/Libre and Open Source Software.
• Tezos is a proof-of-stake distributed consensus platform designed to
evolve, and is written in OCaml. The version of the [Tezos] daemon
that now runs on Multicore OCaml is also available in GitHub as a
work-in-progress fork.
[ocaml-multicore/retro-httpaf-bench#13]
<https://github.com/ocaml-multicore/retro-httpaf-bench/pull/13>
[ocaml-multicore/ocaml-uring#36]
<https://github.com/ocaml-multicore/ocaml-uring/pull/36>
[ocaml-multicore/domainslib#41]
<https://github.com/ocaml-multicore/domainslib/pull/41>
[ParaFuzz] <https://github.com/ocaml-multicore/parafuzz>
[Tezos] <https://github.com/ocaml-multicore/tezos>
Eio
┈┈┈
The `eio' library provides an effects-based parallel IO stack for
Multicore OCaml.
◊ Completed
• [ocaml-multicore/eio#68] Add eio_luv backend
We now use `luv', which has OCaml/Reason bindings to libuv, to
provide a cross-platform default backend for eio.
• [ocaml-multicore/eio#72] Add non-deterministic to abstract domain
socket test
The inclusion of `non-deterministic=command' to disable a regular
`dune runtest' for the failing abstract domain socket test.
• [ocaml-multicore/eio#73] Work-around for `io_uring' bug reading from
terminals
A work-around to fix `IORING_OP_READ' that causes `io_uring_enter'
to block the entire process when reading from a terminal.
• [ocaml-multicore/eio#74] Don't crash when receiving a signal
A patch to receive a signal and not crash in
`lib_eio_linux/eio_linux.ml'.
• [ocaml-multicore/eio#75] Add Eio.Stream
The `Stream' module has been added to Eio that implements bounded
queues with cancellation.
• [ocaml-multicore/eio#76] Link to some eio examples
The README.md has been updated to point to existing eio example
project sources.
• [ocaml-multicore/eio#77] Disable opam file generation due to dune
bug
The opam file generation with dune.2.9.0 is broken as dune does not
have the `subst --root' option. Hence, the same is now disabled in
the eio build steps.
• [ocaml-multicore/eio#79] Initial edits for consistency, formatting
and clarity
Changes in the README.md file for consistency, syntax formatting and
for clarity.
[ocaml-multicore/eio#68]
<https://github.com/ocaml-multicore/eio/pull/68>
[ocaml-multicore/eio#72]
<https://github.com/ocaml-multicore/eio/pull/72>
[ocaml-multicore/eio#73]
<https://github.com/ocaml-multicore/eio/pull/73>
[ocaml-multicore/eio#74]
<https://github.com/ocaml-multicore/eio/pull/74>
[ocaml-multicore/eio#75]
<https://github.com/ocaml-multicore/eio/pull/75>
[ocaml-multicore/eio#76]
<https://github.com/ocaml-multicore/eio/pull/76>
[ocaml-multicore/eio#77]
<https://github.com/ocaml-multicore/eio/pull/77>
[ocaml-multicore/eio#79]
<https://github.com/ocaml-multicore/eio/pull/79>
Benchmarking
╌╌╌╌╌╌╌╌╌╌╌╌
Sandmark
┄┄┄┄┄┄┄┄
• [ocaml-bench/sandmark#251] Update dependencies to work with
4.14.0+trunk
A series of patches that update the dependencies in Sandmark to
build 4.14.0+trunk with dune.2.9.0.
• We are continuing to integrate and test building of 4.12.0 OCaml
variants with Sandmark-2.0 with `current-bench' for both sequential
and parallel benchmarks.
Our thanks to all the OCaml users, developers and contributors in the
community for their valuable time and support to the project. Stay
safe!
[ocaml-bench/sandmark#251]
<https://github.com/ocaml-bench/sandmark/pull/251>
Acronyms
╌╌╌╌╌╌╌╌
• GC: Garbage Collector
• GDB: GNU Project Debugger
• HTTP: Hypertext Transfer Protocol
• ICFP: International Conference on Functional Programming
• IO: Input/Output
• PR: Pull Request
• OPAM: OCaml Package Manager
• VM: Virtual Machine
Higher-kinded bounded polymorphism
══════════════════════════════════
Archive:
<https://sympa.inria.fr/sympa/arc/caml-list/2021-09/msg00005.html>
Oleg announced
──────────────
Abstraction over polymorphic types is something that one comes across
from time to time, especially when implementing operations generic
over collections, or embedding typed DSLs (particularly in
tagless-final style). One immediately learns that OCaml does not have
higher-kinded type variables (type variables that range over type
constructors like list and array); does not permit type constructors
appearing by themselves, without being applied to the right number of
arguments; and restricts polymorphic types in other ways (e.g., not
allowing them in type constraints of package types). One soon learns
that the module system (using functors) is a way to realize
higher-kinded polymorphism in OCaml. It takes longer to learn it is
not the only way, however.
The following web page is written to speed up this learning. It
collects what I have learned and rediscovered on this topic, arranged
into a single story and with new explanations and attributions (and
occasionally, new variations).
<http://okmij.org/ftp/ML/higher-kind-poly.html>
One interesting discovery is that some problems that ostensibly
require higher-kinded polymorphism really do not.
The article comes with the complete accompanying code.
OCaml Workshop 2021: Live Stream
════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-workshop-2021-live-stream/8366/9>
Shakthi Kannan announced
────────────────────────
The direct links (for search results) to the videos of the talks from
the OCaml Workshop at ICFP 2021 are listed below:
• GopCaml A structural Editor for OCaml
(<https://www.youtube.com/watch?v=KipRuiLXYEo>)
• OCaml and Python getting the best of both worlds
(<https://www.youtube.com/watch?v=ZseAgG9kmMw>)
• Adapting the OCaml ecosystem for Multicore OCaml
(<https://www.youtube.com/watch?v=MfjMpznU4nM>)
• Parafuzz coverage guided Property Fuzzing for Multicore OCaml
programs (<https://www.youtube.com/watch?v=GZsUoSaIpIs>)
• Wibbily Wobbly Timey Camly
(<https://www.youtube.com/watch?v=3Jpla-vWLR4>)
• Leveraging Formal Specifications to Generate Fuzzing Suites
(<https://www.youtube.com/watch?v=-6fpLF_n7cA>)
• Probabilistic resource limits using StatMemprof
(<https://www.youtube.com/watch?v=9OES0qe_OvE>)
• Continuous Benchmarking for OCaml Projects
(<https://www.youtube.com/watch?v=folKaWN7jy4>)
• A Multiverse of Glorious Documentation
(<https://www.youtube.com/watch?v=qUMbbKwagaU>)
• Digodoc and Docs (<https://www.youtube.com/watch?v=gODavKhyd2I>)
• Keynote - Xavier Leroy: 25 years of OCaml
(<https://www.youtube.com/watch?v=PGJdHUqgX7o>)
• Experiences with Effects
(<https://www.youtube.com/watch?v=k3oQwpyXmpo>)
• opam-bin Binary Packages with Opam
(<https://www.youtube.com/watch?v=vadoOkgEs6w>)
• Love a readable language interpreted by blockchain
(<https://www.youtube.com/watch?v=um55QNwz8o4>)
• From 2n+1 to n (<https://www.youtube.com/watch?v=3xRGeJSslPE>)
• Property based testing for OCaml through Coq
(<https://www.youtube.com/watch?v=w-dTTlVwVgY>)
• Safe protocol updates via propositional logic
(<https://www.youtube.com/watch?v=ckN1YREM3Gk>)
• Binary Analysis Program (BAP)
(<https://www.youtube.com/watch?v=S9XPI1fHWqM>)
• Semgrep a fast lightweight polyglot static analysis tool to find
bugs (<https://www.youtube.com/watch?v=q7kuvyAOk78>)
Windows-friendly OCaml 4.12 distribution 2nd preview release (0.2.0)
════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-windows-friendly-ocaml-4-12-distribution-2nd-preview-release-0-2-0/8488/1>
jbeckford announced
───────────────────
I am pleased to announce the second preview release of the *Diskuv
OCaml* (“DKML”) distribution. *Diskuv OCaml* is an OCaml distribution
focused on a) secure, cross-platform software development and b) ease
of use for language learners and professional developers.
Thanks to everybody who tried out the first preview! The first preview
was geared towards Windows users and distributed Microsoft compiled
packages. The Microsoft compiler is critical for first-class Windows
support: many popular C libraries like ['libuv' support the Microsoft
compiler as *Tier 1*] but relegate the GCC compiler to an unsupported
third-class citizen.
This second preview has the same Windows users in mind, with several
new features and improvements. There are too many changes to list
here, but some notable ones are:
• (Critical security fix) The MSYS2 installer is now downloaded over
HTTPS and its SHA256 checksum is validated.
• Removed auto-installation of Cygwin and removed the 16GB
auto-download of two Docker images. This cuts down the install time
in half (except for some people; see sharp edges below), and should
reduce the frequency of installation failures caused by limited or
flaky networks.
• Autodetection and use of existing Visual Studio instances on your
machine if the installations are compatible with DKML.
• Clarified that students are covered under free personal licenses in
a prominent location in the main READMEs.
• The ctypes, mirage-crypto and feather packages are available for
testing in a *highly unstable* form. That means support may be
deprecated in a future release, but today they all build + pass
tests with the Microsoft compiler.
There are also some sharp edges in this preview. The major ones are:
• On some machines (especially Docker containers or VMs) one of the
installation commands can peg your CPU for up to 2 hours while
making incredibly slow progress.
• There are backwards incompatible changes. See the release notes
linked below.
[The Version 0.2.0 release notes] has instructions for upgrading and
also a detailed list of changes. Instructions for installing from
scratch and updated documentation are still at the [Diskuv OCaml
documentation site].
*What's Next?*
Barring some critical security updates, the next preview will focus on
Linux and macOS parity, and the removal of the sharp edges. Will post
another "[ANN]" and cross-link on Twitter when that preview is ready.
In the meantime, I would appreciate feedback!
['libuv' support the Microsoft compiler as *Tier 1*]
<https://github.com/libuv/libuv/blob/v1.x/SUPPORTED_PLATFORMS.md#supported-platforms>
[The Version 0.2.0 release notes]
<https://gitlab.com/diskuv/diskuv-ocaml/-/releases/v0.2.0#020-2021-09-13>
[Diskuv OCaml documentation site]
<https://diskuv.gitlab.io/diskuv-ocaml/>
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <https://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <https://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20210914/372eaa26/attachment-0001.html>
More information about the caml-news-weekly
mailing list