[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Jul 13 06:55:29 PDT 2021
Hello
Here is the latest OCaml Weekly News, for the week of July 06 to 13,
2021.
Table of Contents
─────────────────
slug 1.0.0 - URL-safe slug generator
Developer Experience Engineer at Jane Street
Hardcaml MIPS CPU Learning Project and Blog
OCaml for Windows installation confusion
OCamlFormat config file auto-completion support in VSCode
Multicore OCaml: June 2021
memprof-limits (first official release): Memory limits, allocation limits, and thread cancellation, with interrupt-safe resources
Bitwuzla 1.0.0 (SMT solver for AUFBVFP)
Old CWN
slug 1.0.0 - URL-safe slug generator
════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-slug-1-0-0-url-safe-slug-generator/8107/1>
Khoa Nguyen announced
─────────────────────
I want to introduce my first library ever released on opam. Link to
Github repository: <https://github.com/thangngoc89/ocaml-slug>
ocaml-slug
╌╌╌╌╌╌╌╌╌╌
Url safe slug generator for OCaml
A URL slug is the part of a URL or link that comes after
the domain extension. In websites the keyword used for
your URL slug can be used to SEO optimize the URL by
showing Google the structure of your site and the contents
of the page in question.
This library turns title into URL-safe slug with support for non-latin
characters.
This library uses algorithm and data from [node-slugify]
[node-slugify] <https://github.com/simov/slugify>
Installation
┄┄┄┄┄┄┄┄┄┄┄┄
┌────
│ opam install slug
└────
If you use [esy]
┌────
│ esy add @opam/slug
└────
[esy] <https://esy.sh>
Usage
┄┄┄┄┄
• OCaml syntax:
┌────
│ Slug.slugify "my string";;
│ - : string = "my-string"
│
│ (* Custom separator *)
│ Slug.slugify ?sep: "_" "my string";;
│ - : string = "my_string"
│
│ (* Retain uppercase *)
│ Slug.slugify ?lowercase: false "My String";;
│ - : string = "My-String"
│
│ (* Use locale *)
│ let with_vi = Slug.(Charmap.mk_charmap [Slug_data.base; Slug_data.vi]);;
│ - : Charmap.t = <abstr>
│ Slug.slugify ?charmap: with_vi "Đ";;
│ - : string = "d"
│ Slug.slugify "Đ";;
│ - : string = "dj"
│
│ (* Custom characters map *)
│ let custom_map = Slug.(Charmap.mk_charmap [Slug_data.base; [ ("M", "z"); ("m", "z") ]]);;
│ val custom_map : Charmap.t = <abstr>
│
│ Slug.slugify ?charmap: custom_map "Mm";;
│ - : string = "zz"
└────
• ReasonML syntax
┌────
│ Slug.slugify("my string");
│ - : string = "my-string"
│
│ /* Custom separator */
│ Slug.slugify(~sep = "_", "my string");
│ - : string = "my_string"
│
│ /* Retain uppercase */
│ Slug.slugify(~lowercase = false, "My String");
│ - : string = "My-String"
│
│ /* Use locale */
│ let with_vi = Slug.(Charmap.mk_charmap([Slug_data.base, Slug_data.vi]));
│ let with_vi : Charmap.t = <abstr>
│ Slug.slugify(~charmap = with_vi, "Đ");
│ - : string = "d"
│ Slug.slugify("Đ");
│ - : string = "dj"
│
│ /* Custom characters map */
│ let custom_map = Slug.(Charmap.mk_charmap([Slug_data.base, [ ("M", "z"), ("m", "z") ]]));;
│ let custom_map : Charmap.t = <abstr>
│
│ Slug.slugify(~charmap = custom_map, "Mm");
│ - : string = "zz"
└────
Notice
┄┄┄┄┄┄
Please don't send PR to update `data/*'. They are auto-generated from
[upstream library].
Please send PRs about new locales to node-slugify.
If you really need it, you can use a `custom_map' instead.
[upstream library] <https://github.com/simov/slugify>
License
┄┄┄┄┄┄┄
MIT. Data are downloaded from [node-slugify]
[node-slugify] <https://github.com/simov/slugify>
Developer Experience Engineer at Jane Street
════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/job-developer-experience-engineer-at-jane-street/8113/1>
Yaron Minsky announced
──────────────────────
You can read more about the job here:
<https://blog.janestreet.com/looking-for-a-developer-experience-engineer-index/>
The role is for someone to act as an advocate for our internal
developers, helping move information in both directions: teaching
people how to make most use of the tools, and figuring out what our
developers need, and helping set priorities for our dev-tools and
compiler teams.
Please share this with anyone you think might be a good fit for the
role!
Hardcaml MIPS CPU Learning Project and Blog
═══════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/hardcaml-mips-cpu-learning-project-and-blog/8088/9>
Clément asked and Alexander (Sasha) Skvortsov announced
───────────────────────────────────────────────────────
👍 Does your website have an RSS feed?
It does now! `https://ceramichacker.com/rss'. Also
`https://ceramichacker.com/atom' for Atom users.
On a side note, we've published two more posts:
1. [Memory in Hardcaml]
2. [Registers and Stateful Design]
More coming soon!
[Memory in Hardcaml]
<https://ceramichacker.com/blog/12-6x-memory-in-hardcaml>
[Registers and Stateful Design]
<https://ceramichacker.com/blog/13-7x-registers-and-stateful-design>
OCaml for Windows installation confusion
════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-for-windows-installation-confusion/8105/17>
Deep in this thread, Nicolás Ojeda Bär said
───────────────────────────────────────────
At LexiFi we have been writing OCaml under Windows for a long time and
are quite experienced with it. I know of at least one other large
industrial player that uses OCaml on Windows as their main environment
cc @keleshev.
If I had to summarize the situation of OCaml on Windows I would say
that once you get past the installation of the dev environment (C
toolchain + Cygwin), the experience is quite similar to Linux. The
compiler, standard library, `dune' and `merlin', all work flawlessly
on Windows. OPAM is not 100% ready yet on Windows (we don't use it)
but it has never been easier to work "monorepo" style using `dune'.
For beginners the main stumbling block is setting up the dev
environment (C toolchain + Cygwin). On Linux installing a C toolchain
is not needed because the compiler is installed by default, but this
is not the case on Windows. As for Cygwin, it is strictly speaking
only necessary when building the compiler itself. Once the compiler is
installed you are free to never use Cygwin again…
So if you want a Rust-style exeperience for beginners you need to
figure out how to provide a point-and-click installer that does the
following things:
1. installs the C toolchain,
2. installs Cygwin and builds & installs the OCaml compiler using it
3. installs merlin and dune
At this point you will be left with a very capable environment for
writing OCaml programs using common editors such as VS Code.
Regarding 1. above (the C toolchain), there are two main options. If
you want a completely freestanding C toolchain you *must* use the
native MSVC compiler command-line tools, which you can get from
<https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line>
(link appears down at the moment).
If you are willing to use Cygwin for your day-to-day developing (as
opposed to only using it when building the compiler), then you have a
second option which is to use the `mingw64' compiler. This is a
compiler that produces native Windows binaries, but the advantage is
that it is essentially `gcc' so you get an user experience which you
may be used to from Linux. As it runs under Cygwin, this option may
appeal to those that prefer to insulate themselves as much as possible
from Windows specificities.
My guess is that the recommanded windows way is WSL2.
It depends what you mean by "on Windows". If all you want is to
develop "on" a Windows machine, yes by all means WSL is a good
option. But the binaries you produce are Linux binaries, so they won't
run on Windows outside of the WSL environment.
Thank you. However if the repository won’t get any more
updates from next month, that seems like a showstopper for
OCaml usage on Windows to me.
Only if you insist on using OPAM for your development. It is perfectly
feasible to develop on Windows "monorepo" style without using
OPAM. The experience may not be as pleasent as it would be on Unix and
depending on how much you rely on external libraries it may be more or
less convenient, but it works quite well in general.
OCamlFormat config file auto-completion support in VSCode
═════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocamlformat-config-file-auto-completion-support-in-vscode/8118/1>
тars announced
──────────────
Hi, I made a tiny vscode extension which provides auto-completion for
`.ocamlformat' file. (I posted this on discord but let me post it here
too.)
…BTW I'm *a complete beginner* so this is written in TS. If I become
comfortable with OCaml and its ecosystems, I'd like to re-write this
in OCaml. But as of now, I have no clue. :upside_down_face:
Cheers.
<https://github.com/tars0x9752/ocamlformat-auto-completion-vscode>
тars then added
───────────────
Here's how it works.
<https://user-images.githubusercontent.com/46079709/124963946-666d0a80-e05b-11eb-9747-75002e458818.gif>
Max Lantas replied
──────────────────
Wow, this is great! Nice job!
I’d like to re-write this in OCaml.
If you decide to try this, you can look [VSCode OCaml Platform source
code] to see how we wrote the OCaml extension in OCaml code. The basic
idea is that it uses the [Js_of_ocaml compiler] to compile OCaml code
to Javascript with bindings to the VSCode API.
I glanced over your source code and it looks like it would translate
very well to OCaml code. Feel free to reach out if you'd like help.
[VSCode OCaml Platform source code]
<https://github.com/ocamllabs/vscode-ocaml-platform>
[Js_of_ocaml compiler] <https://github.com/ocsigen/js_of_ocaml>
Multicore OCaml: June 2021
══════════════════════════
Archive:
<https://discuss.ocaml.org/t/multicore-ocaml-june-2021/8134/1>
Anil Madhavapeddy announced
───────────────────────────
Welcome to the June 2021 [Multicore OCaml] monthly report! This
month's update along with the [previous update's] have been compiled
by @avsm, @ctk21, @kayceesrk and @shakthimaan.
Our overall goal remains on track for generating a preview tree for
OCaml 5.0 multicore domains-only parallelism over the summer.
[Multicore OCaml] <https://github.com/ocaml-multicore/ocaml-multicore>
[previous update's] <https://discuss.ocaml.org/tag/multicore-monthly>
Ecosystem compatibility for 4.12.0+domains
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
In [May's update], I noted that our focus was now on adapting the
ecosystem to work well with multicore, and I'm pleased to report that
this is progressing very well.
• The 4.12.0+domains multicore compiler variant has been [merged into
mainline opam-repo], so you can now `opam switch 4.12.0+domains'
directly. The `base-domains' package is also available to mark your
opam project as _requiring_ the `Domains' module, so you can even
publish your early multicore-capable libraries to the mainline opam
repository now.
• The OCaml standard library was made safe for parallel use by
multiple domains ([wiki], [issue], [fixes]); and in particularly the
`Format' and `Random' modules. These modules were the main sources
of incompatibilities we found when running existing OCaml code with
multiple domains.
• The `Domain' module has had its interface slimmed with the removal
of `critical_section', `wait', `notify' which has allowed
significant runtime simplification. The GC C-API interface is now
implemented and this means that Jane Street's `Base', `Core', and
`Async' now compile on `4.12+domains' without modifications; for
example `opam install patdiff' works out of the box on a
`4.12+domains' switch!
• [Domainslib 0.3.0] has been released which incorporates multiple
improvements including the work-stealing deques for task
distribution. The performance of reading domain local variables has
also been improved with a primitive and a O(1) lookup. The chapter
on [`Parallel Programming in Multicore OCaml'] has been updated to
reflect the latest developments with Domainslib.
This means that big application stacks should now compile pretty well
with 4.12.0+domains (applications like the Tezos node and patdiff
exercise a lot of the dependency trees in opam). If you do find
incompatibilities, please do report them on the [repository].
[May's update]
<https://discuss.ocaml.org/t/multicore-ocaml-may-2021/7990#ecosystem-changes-to-prepare-for-500-domains-only-2>
[merged into mainline opam-repo]
<https://github.com/ocaml/opam-repository/pull/18960>
[wiki]
<https://github.com/ocaml-multicore/ocaml-multicore/wiki/Safety-of-Stdlib-under-Multicore-OCaml>
[issue] <https://github.com/ocaml/ocaml/issues/10453>
[fixes]
<https://github.com/ocaml-multicore/ocaml-multicore/issues?q=is%3Aissue+label%3A%22stdlib+safety%22+is%3Aclosed>
[Domainslib 0.3.0]
<https://github.com/ocaml-multicore/domainslib/releases/tag/0.3.0>
[`Parallel Programming in Multicore OCaml']
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml>
[repository] <https://github.com/ocaml-multicore/ocaml-multicore/issues>
4.12.0+domains+effects
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Most of our focus has been on getting the domains-only trees (for
OCaml 5.0) up to speed, but we have been progressing the direct-style
effects-based IO stack as well.
• The `uring' bindings to Linux Io_uring are now available on
opam-repository, so you can try it out on sequential OCaml too. A
good mini-project would be to add a uring backend to the existing
Async or Lwt engines, if anyone wants to try a substantial
contribution.
• The [`eio' library] is fairly usable now, for both filesystem and
networking. We've submitted a talk to the OCaml workshop to dive
into the innards of it in more detail, so stay tuned for that in the
coming months if accepted. The main changes here have been
performance improvements, and the HTTP stack is fairy competitive
with (e.g.) `rust-hyper'.
We will soon also have a variant of this tree that removes the custom
effect syntax and implements the fibres (the runtime piece) as `Obj'
functions. This will further improve ecosystem compatibility and
allow us to build direct-style OCaml libraries that use fibres
internally to provide concurrency, but without exposing any use of
effects in their interfaces.
[`eio' library] <https://github.com/ocaml-multicore/eio>
Benchmarking and performance
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
We are always keen to get more benchmarks that exercise multicore
features; if you want to try multicore out and help write benchmarks
there are some suggestions on the [wiki]. We've got a private server
which runs a Sandmark nightly benchmark pipeline with Jupyter
notebooks, which we can give access to anyone who submits
benchmarks. We continue to test integration of Sandmark with
[current-bench] for better integration with GitHub PRs.
As always, the Multicore OCaml ongoing and completed tasks are listed
first, which are then followed by updates from the ecosystem and their
associated libraries. The Sandmark benchmarking and nightly build
efforts are then mentioned. Finally, the status of the upstream OCaml
Safepoints PR is provided for your reference.
[wiki]
<https://github.com/ocaml-multicore/ocaml-multicore/wiki/Multicore-benchmarking-projects>
[current-bench] <https://github.com/ocurrent/current-bench>
Multicore OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml-multicore/ocaml-multicore#573] Backport trunk safepoints PR
to multicore
A work-in-progress to backport the Safepoints PR from ocaml/ocaml to
Multicore OCaml.
• [ocaml-multicore/ocaml-multicore#584] Modernise signal handling
A patch to bring the Multicore OCaml signals implementation closer
to upstream OCaml.
• [ocaml-multicore/ocaml-multicore#598] Do not deliver signals to
threads that have blocked them
A draft PR to not deliver signals to threads that are in a blocked
state. The without-systhreads case needs to be handled.
• [ocaml-multicore/ocaml-multicore#600] Expose a few more GC variables
in headers
The `caml_young_start', `caml_young_limit' and `caml_minor_heap_wsz'
variables have been defined in the runtime.
• [ocaml-multicore/ocaml-multicore#601] Domain better participants
The iterations `0(Max_domains)' from STW signalling and
`0(n_running_domains)' from domain creation have now been removed.
• [ocaml-multicore/ocaml-multicore#603] Systhreads tick thread
An initial draft PR for porting the tick thread to Multicore OCaml.
[ocaml-multicore/ocaml-multicore#573]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/573>
[ocaml-multicore/ocaml-multicore#584]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/584>
[ocaml-multicore/ocaml-multicore#598]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/598>
[ocaml-multicore/ocaml-multicore#600]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/600>
[ocaml-multicore/ocaml-multicore#601]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/601>
[ocaml-multicore/ocaml-multicore#603]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/603>
Completed
┄┄┄┄┄┄┄┄┄
Enhancements
┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#552] Add a
`force_instrumented_runtime' option to configure
The `configure' script now accepts a new
`--enable-force-instrumented-runtime' option to facilitate use of
the instrumented runtime on linker invocations to obtain event logs.
• [ocaml-multicore/ocaml-multicore#558] Refactor `Domain.{spawn/join}'
to use no critical sections
The critical sections in `Domain.{spawn/join}' and the use of
`Domain.wait' have been removed.
• [ocaml-multicore/ocaml-multicore#561] Slim down `Domain.Sync':
remove `wait', `notify', `critical_section'
A breaking change in `Domain.Sync' that removes `critical_section',
`notify', `wait', `wait_for', and `wait_until'. This is to remove
the need for domain-to-domain messaging in the runtime.
• [ocaml-multicore/ocaml-multicore#576] Including Git hash in runtime
A Git hash is now printed in the runtime as shown below:
┌────
│ $ ./boot/ocamlrun -version
│ The OCaml runtime, version 4.12.0+multicore
│ Built with git hash 'ae3fb4bb6' on branch 'runtime_version' with tag '<tag unavailable>'
└────
• [ocaml-multicore/ocaml-multicore#579] Primitive for fetching DLS
root
A new primitive has been implemented for fetching DLS, and is now a
single `mov' instruction on `amd64'.
[ocaml-multicore/ocaml-multicore#552]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/552>
[ocaml-multicore/ocaml-multicore#558]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/558>
[ocaml-multicore/ocaml-multicore#561]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/561>
[ocaml-multicore/ocaml-multicore#576]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/576>
[ocaml-multicore/ocaml-multicore#579]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/579>
Upstream
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#555] runtime: `CAML_TRACE_VERSION'
is now set to a Multicore specific value
A `CAML_TRACE_VERSION' is defined to distinguish between Multicore
OCaml and trunk for the runtime.
• [ocaml-multicore/ocaml-multicore#581] Move our usage of inline to
`Caml_inline'
We now use `Caml_inline' for all the C inlining in the runtime to
align with upstream OCaml.
• [ocaml-multicore/ocaml-multicore#589] Reintroduce `adjust_gc_speed'
The `caml_adjust_gc_speed' function from trunk has been reintroduced
to the Multicore OCaml runtime.
• [ocaml-multicore/ocaml-multicore#590] runtime: stub `caml_stat_*'
interfaces in gc_ctrl
The creation of `caml_stat_*' stub functions in gc_ctrl.h to
introduce a compatibility layer for GC stat utilities that are
available in trunk.
[ocaml-multicore/ocaml-multicore#555]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/555>
[ocaml-multicore/ocaml-multicore#581]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/581>
[ocaml-multicore/ocaml-multicore#589]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/589>
[ocaml-multicore/ocaml-multicore#590]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/590>
Fixes
┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#562] Import fixes to the minor heap
allocation code from DLABs
The multiplication factor of two used for minor heap allocation has
been removed, and the `Minor_heap_max' limit from config.h is no
longer converted to a byte size for Multicore OCaml.
• [ocaml-multicore/ocaml-multicore#593] Fix two issues with ephemerons
A patch to simplify ephemeron handover during termination.
• [ocaml-multicore/ocaml-multicore#594] Fix finaliser handover issue
The `caml_finish_major_cycle' is used leading to the major GC phase
`Phase_sweep_and_mark_main' for the correct handoff of finalisers.
• [ocaml-multicore/ocaml-multicore#596] systhreads: do `st_thread_id'
after initializing the thread descriptor
The thread ID was set even before initializing the thread
descriptor, and this PR fixes the order.
• [ocaml-multicore/ocaml-multicore#604] Fix unguarded
`caml_skiplist_empty' in `caml_scan_global_young_roots'
The PR introduces a `caml_iterate_global_roots' function and fixes a
locking bug with global roots.
[ocaml-multicore/ocaml-multicore#562]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/562>
[ocaml-multicore/ocaml-multicore#593]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/593>
[ocaml-multicore/ocaml-multicore#594]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/594>
[ocaml-multicore/ocaml-multicore#596]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/596>
[ocaml-multicore/ocaml-multicore#604]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/604>
Cleanups
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#567] Simplify some of the minor_gc
code
The `not_alone' variable has been cleaned up with a simplification
to the minor_gc.c code.
• [ocaml-multicore/ocaml-multicore#580] Remove struct domain
The `caml_domain_state' is now the single source of domain
information with the removal of `struct domain'. `struct
dom_internal' is no longer leaking across the runtime.
• [ocaml-multicore/ocaml-multicore#583] Removing interrupt queues
The locking of `struct_interruptor' when receiving interrupts and
the use of `struct interrupt' have been removed, simplifying the
implementation of domains.
[ocaml-multicore/ocaml-multicore#567]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/567>
[ocaml-multicore/ocaml-multicore#580]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/580>
[ocaml-multicore/ocaml-multicore#583]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/583>
Sundries
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#582] Make global state domain-local
in Random, Hashtbl and Filename
The Domain-Local is now set as the default state in `Random',
`Hashtbl' and `Filename'.
• [ocaml-multicore/ocaml-multicore#586] Make the state in Format
domain-local
The default state in `Format' is now set to Domain-Local.
• [ocaml-multicore/ocaml-multicore#595] Implement
`caml_alloc_dependent_memory' and `caml_free_dependent_memory'
Dependent memory are the blocks of heap memory that depend on the GC
(and finalizers) for deallocation. The `caml_alloc_dependent_memory'
and `caml_free_dependent_memory' have been added to
runtime/memory.c.
[ocaml-multicore/ocaml-multicore#582]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/582>
[ocaml-multicore/ocaml-multicore#586]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/586>
[ocaml-multicore/ocaml-multicore#595]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/595>
Ecosystem
╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml-multicore/eventlog-tools#3] Use ocaml/setup-ocaml at v2
An update to `.github/workflows/main.yml' to build for
ocaml/setup-ocaml at v2.
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#7] Add a
section on Domain-Local Storage
The README.md file now includes a section on Domain-Local Storage.
• [ocaml-multicore/eio#26] Grand Central Dispatch Backend
The implemention of the Grand Central Dispatch (GCD) backend for Eio
is a work-in-progress.
• [ocaml-multicore/domainslib#34] Fix initial value accounting in
`parallel_for_reduce'
A patch to fix the initial value in `parallel_for_reduce' as it was
being accounted for multiple times.
• [ocaml-multicore/domainslib#36] Switch to default `Random' module
The library has been updated to use the default `Random' module as
it stores its state in Domain-Local Storage which can be called from
multiple domains. The Sandmark results are given below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/9/9a68704f282ac07167fb7d909334befa437593f0.png>
• [ocaml-multicore/multicore-opam#56] Base-effects depends strictly on
4.12
A query on the use of strict 4.12.0 lower bound for OCaml in
`base-effects.base/opam'.
• [ocsigen/lwt#860] Lwt_domain: An interfacet to Multicore parallelism
The `Lwt_domain' module has been ported to domainslib Task pool for
performing computations to CPU cores using Multicore OCaml's
Domains. A few benchmark results obtained on an Intel Xeon Gold 5120
processor with 24 isolated cores is shown below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/1/1e2d018635857f3603a1b7f9573176840efabede.png>
[ocaml-multicore/eventlog-tools#3]
<https://github.com/ocaml-multicore/eventlog-tools/pull/3>
[ocaml-multicore/parallel-programming-in-multicore-ocaml#7]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/7>
[ocaml-multicore/eio#26]
<https://github.com/ocaml-multicore/eio/pull/26>
[ocaml-multicore/domainslib#34]
<https://github.com/ocaml-multicore/domainslib/pull/34>
[ocaml-multicore/domainslib#36]
<https://github.com/ocaml-multicore/domainslib/pull/36>
[ocaml-multicore/multicore-opam#56]
<https://github.com/ocaml-multicore/multicore-opam/issues/56>
[ocsigen/lwt#860] <https://github.com/ocsigen/lwt/pull/860>
Completed
┄┄┄┄┄┄┄┄┄
Ocaml-Uring
┈┈┈┈┈┈┈┈┈┈┈
The `ocaml-uring' repository contains bindings to `io_uring' for
OCaml.
• [ocaml-multicore/ocaml-uring#21] Add accept call
The `accept' call has been added to uring along with the inclusion
of the `unix' library as a dependency.
• [ocaml-multicore/ocaml-uring#22] Add support for cancellation
A `cancel' method is added to request jobs for cancellation. The
queuing operations and tests have also been updated.
• [ocaml-multicore/ocaml-uring#24] Sort out cast
The `Int_val' has been changed to `Long_val' to remove the need for
sign extension instruction on 64-bit platforms.
• [ocaml-multicore/ocaml-uring#25] Fix test_cancel
A `with_uring' function is added with a `queue_depth' argument to
handle tests for cancellation.
• [ocaml-multicore/ocaml-uring#26] Add `openat2'
The `openat2' method has been added giving access to all the Linux
open and resolve flags.
• [ocaml-multicore/ocaml-uring#27] Fine-tune C flags for better
performance
The CFLAGS have been updated for performance improvements. The
following results are observed for the noop benchmark:
┌────
│ Before: noop 10000 │ 1174227.1170 ns/run│
│ After: noop 10000 │ 920622.5802 ns/run│
└────
• [ocaml-multicore/ocaml-uring#28] Don't allow freeing the ring while
it is in use
The ring is added to a global set on creation and is cleaned up on
exit. Also, invalid cancellation requests are checked before
allocating a slot.
• [ocaml-multicore/ocaml-uring#29] Replace iovec with cstruct and
clean up the C stubs
The `readv' and `writev' now accept a list of Cstructs which allow
access to sub-ranges of bigarrays, and to work with multiple
buffers. The handling of OOM errors has also been improved.
• [ocaml-multicore/ocaml-uring#30] Fix remaining TODOs in API
The `read' and `write' methods have been renamed to `read_fixed' and
`write_fixed' respectively. The `Region.to_cstruct' has been added
as an alternative to creating a sub-bigarray. An exception is now
raised if the user requests for a larger size chunk.
• [ocaml-multicore/ocaml-uring#31] Use `caml_enter_blocking_section'
when waiting
The `caml_enter_blocking_section' and `caml_leave_blocking_section'
are used when waiting, which allows other threads to execute and the
GC can run in the case of Multicore OCaml.
• [ocaml-multicore/ocaml-uring#32] Compile `uring' using the C flags
from OCaml
Use the OCaml C flags when building uring, and remove the unused
dune file.
• [ocaml-multicore/ocaml-uring#33] Prepare release
The CHANGES.md, README.md, dune-project and uring.opam files have
been updated to prepare for a release.
• [ocaml-multicore/ocaml-uring#34] Convert `liburing' to subtree
We now use a subtree instead of a submodule so that the ocaml-uring
can be submitted to the opam-repository.
[ocaml-multicore/ocaml-uring#21]
<https://github.com/ocaml-multicore/ocaml-uring/pull/21>
[ocaml-multicore/ocaml-uring#22]
<https://github.com/ocaml-multicore/ocaml-uring/pull/22>
[ocaml-multicore/ocaml-uring#24]
<https://github.com/ocaml-multicore/ocaml-uring/pull/24>
[ocaml-multicore/ocaml-uring#25]
<https://github.com/ocaml-multicore/ocaml-uring/pull/25>
[ocaml-multicore/ocaml-uring#26]
<https://github.com/ocaml-multicore/ocaml-uring/pull/26>
[ocaml-multicore/ocaml-uring#27]
<https://github.com/ocaml-multicore/ocaml-uring/pull/27>
[ocaml-multicore/ocaml-uring#28]
<https://github.com/ocaml-multicore/ocaml-uring/pull/28>
[ocaml-multicore/ocaml-uring#29]
<https://github.com/ocaml-multicore/ocaml-uring/pull/29>
[ocaml-multicore/ocaml-uring#30]
<https://github.com/ocaml-multicore/ocaml-uring/pull/30>
[ocaml-multicore/ocaml-uring#31]
<https://github.com/ocaml-multicore/ocaml-uring/pull/31>
[ocaml-multicore/ocaml-uring#32]
<https://github.com/ocaml-multicore/ocaml-uring/pull/32>
[ocaml-multicore/ocaml-uring#33]
<https://github.com/ocaml-multicore/ocaml-uring/pull/33>
[ocaml-multicore/ocaml-uring#34]
<https://github.com/ocaml-multicore/ocaml-uring/pull/34>
Parallel Programming in Multicore OCaml
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#5]
`num_domains' to `num_additional_domains'
The documentation and code examples have been updated to now use
`num_additional_domains' instead of `num_domains'.
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#6] Update
latest information about compiler versions
The compiler versions in the README.md have been updated to use 4.12
and its variants.
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#8] Nudge
people to the default chunk_size setting
The recommendation is to use the default `chunk_size' when using
`parallel_for', especially when the number of domains gets larger.
• [ocaml-multicore/parallel-programming-in-multicore-ocaml#9] Eventlog
section updates
The `eventlog-tools' library can now be used for parsing trace files
since Multicore OCaml includes CTF tracing support from trunk. The
relevant information has been updated in the README.md file.
[ocaml-multicore/parallel-programming-in-multicore-ocaml#5]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/5>
[ocaml-multicore/parallel-programming-in-multicore-ocaml#6]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/6>
[ocaml-multicore/parallel-programming-in-multicore-ocaml#8]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/8>
[ocaml-multicore/parallel-programming-in-multicore-ocaml#9]
<https://github.com/ocaml-multicore/parallel-programming-in-multicore-ocaml/pull/9>
Eio
┈┈┈
The `eio' library provides an effects-based parallel IO stack for
Multicore OCaml.
◊ Additions
• [ocaml-multicore/eio#41] Add eio.mli file
A `lib_eio/eio.mli' file containing modules for `Generic', `Flow',
`Network', and `Stdenv' have been added to the repository.
• [ocaml-multicore/eio#45] Add basic domain manager
The PR allows you to run a CPU-intensive task on another domain, and
adds a mutex to `traceln' to avoid overlapping output.
• [ocaml-multicore/eio#46] Add Eio.Time and allow cancelling sleeps
Use `psq' instead of `bheap' library to allow cancellations. The
`Eio.Time' module has been added to `lib_eio/eio.ml'.
• [ocaml-multicore/eio#53] Add `Switch.sub_opt'
A new `Switch.sub_opt' implementation has been added to allow
running a function with a new switch. Also, `Switch.sub' has been
modified so that it is not a named argument.
• [ocaml-multicore/eio#54] Initial FS abstraction
A module `Dir' has been added to allow file system abstraction along
with the ability to create files and directories. On Linux, it uses
`openat2' and `RESOLVE_BENEATH'.
• [ocaml-multicore/eio#56] Add `with_open_in', `with_open_out' and
`with_open_dir' helpers
The `Eio.Dir' module now contains a `with_open_in', `with_open_out'
and `with_open_dir' helper functions.
• [ocaml-multicore/eio#58] Add `Eio_linux.{readv, writev}'
The `Eio_linux.{readv, writev}' functions have been added to
`lib_eio_linux/eio_linux.ml' which uses the new OCaml-Uring API.
• [ocaml-multicore/eio#59] Add `Eio_linux.noop' and a simple benchmark
A `Eio_linux.noop' implementation has been added for benchmarking
Uring dispatch.
• [ocaml-multicore/eio#61] Add generic Enter effect to simplify
scheduler
A `Enter' effect has been introduced to simplify the scheduler
operations, and this does not have much effect on the noop benchmark
as illustrated below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/3/3c1b2df002e1f57be2850d8b81d5eb08afb097dd.png>
[ocaml-multicore/eio#41]
<https://github.com/ocaml-multicore/eio/pull/41>
[ocaml-multicore/eio#45]
<https://github.com/ocaml-multicore/eio/pull/45>
[ocaml-multicore/eio#46]
<https://github.com/ocaml-multicore/eio/pull/46>
[ocaml-multicore/eio#53]
<https://github.com/ocaml-multicore/eio/pull/53>
[ocaml-multicore/eio#54]
<https://github.com/ocaml-multicore/eio/pull/54>
[ocaml-multicore/eio#56]
<https://github.com/ocaml-multicore/eio/pull/56>
[ocaml-multicore/eio#58]
<https://github.com/ocaml-multicore/eio/pull/58>
[ocaml-multicore/eio#59]
<https://github.com/ocaml-multicore/eio/pull/59>
[ocaml-multicore/eio#61]
<https://github.com/ocaml-multicore/eio/pull/61>
◊ Improvements
• [ocaml-multicore/eio#38] Rename Flow.write to Flow.copy
The code and documentation have been updated to rename `Flow.write'
to `Flow.copy' for better clarity.
• [ocaml-multicore/eio#36] Use uring for accept
The `enqueue_accept' function now uses `Uring.accept' along with the
`effect Accept'.
• [ocaml-multicore/eio#37] Performance improvements
Optimisation for `Eunix.free' and process completed events with
`Uring.peek' for better performance results.
• [ocaml-multicore/eio#48] Simplify `Suspend' operation
The `Suspend' effect has been simplified by replacing the older
`Await' and `Yield' effects with the code from Eio.
• [ocaml-multicore/eio#52] Split Linux support out to `eio_linux'
library
`eunix' now has common code that is shared by different backends,
and `eio_linux' provides a Linux io-uring backend. The tests and the
documentation have been updated to reflect the change.
• [ocaml-multicore/eio#57] Reraise exceptions with backtraces
Added support to store a reference to a backtrace when a switch
catches an exception. This is useful when you want to reraise the
exception later.
• [ocaml-multicore/eio#60] Simplify handling of completions
The PR adds `Job' and `Job_no_cancel' in `type io_job' along with
additional `Log.debug' messages.
[ocaml-multicore/eio#38]
<https://github.com/ocaml-multicore/eio/pull/38>
[ocaml-multicore/eio#36]
<https://github.com/ocaml-multicore/eio/pull/36>
[ocaml-multicore/eio#37]
<https://github.com/ocaml-multicore/eio/pull/37>
[ocaml-multicore/eio#48]
<https://github.com/ocaml-multicore/eio/pull/48>
[ocaml-multicore/eio#52]
<https://github.com/ocaml-multicore/eio/pull/52>
[ocaml-multicore/eio#57]
<https://github.com/ocaml-multicore/eio/pull/57>
[ocaml-multicore/eio#60]
<https://github.com/ocaml-multicore/eio/pull/60>
◊ Cleanups
• [ocaml-multicore/eio#42] Merge fibreslib into eio
The `Fibreslib' code is now merged with `eio'. You will now need to
open `Eio.Std' instead of opening `Fibreslib'.
• [ocaml-multicore/eio#47] Clean up the network API
The network APIs have been updated with few changes such as renaming
`bind' to `listen', replacing `Unix.shutdown_command' with our own
type in Eio API, and replacing `Unix.sockaddr' with a custom type.
• [ocaml-multicore/eio#49] Remove `Eio.Private.Waiters' and
`Eio.Private.Switch'
The `Eio.Private.Waiters' and `Eio.Private.Switch' modules have been
removed, and waiting is now handled using the Eio library.
• [ocaml-multicore/eio#55] Some API and README cleanups
The PR has multiple cleanups and documentation changes. The
README.md has been modified to use `Eio.Flow.shutdown' instead of
`Eio.Flow.close', and a Time section has been added. The
`Eio.Network' module has been changed to `Eio.Net'. The `Time.now'
and `Time.sleep_until' methods have been added to `lib_eio/eio.ml'.
[ocaml-multicore/eio#42]
<https://github.com/ocaml-multicore/eio/pull/42>
[ocaml-multicore/eio#47]
<https://github.com/ocaml-multicore/eio/pull/47>
[ocaml-multicore/eio#49]
<https://github.com/ocaml-multicore/eio/pull/49>
[ocaml-multicore/eio#55]
<https://github.com/ocaml-multicore/eio/pull/55>
◊ Documentation
• [ocaml-multicore/eio#43] Add design note about determinism
The README.md documentation has been updated with few design notes
on Determinism.
• [ocaml-multicore/eio#50] README improvements
Updated README.md and added `doc/prelude.ml' for use with MDX.
[ocaml-multicore/eio#43]
<https://github.com/ocaml-multicore/eio/pull/43>
[ocaml-multicore/eio#50]
<https://github.com/ocaml-multicore/eio/pull/50>
Handling Cancellation
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/eio#39] Allow cancelling accept operations
The PR now supports cancelling the server accept and read
operations.
• [ocaml-multicore/eio#40] Support cancelling the remaining Uring
operations
The cancellation request of `connect', `wait_readable' and
`await_writable' Uring operations is now supported.
• [ocaml-multicore/eio#44] Fix read-cancel test
The `ENOENT' value has been correctly fixed to use -2, and the
documentation for cancelling the read request has been updated.
• [ocaml-multicore/eio#51] Getting `EALREADY' from cancel is not an
error
Handle `EALREADY' case in `lib_eunix/eunix.ml' where an operation
got cancelled while in progress.
[ocaml-multicore/eio#39]
<https://github.com/ocaml-multicore/eio/pull/39>
[ocaml-multicore/eio#40]
<https://github.com/ocaml-multicore/eio/pull/40>
[ocaml-multicore/eio#44]
<https://github.com/ocaml-multicore/eio/pull/44>
[ocaml-multicore/eio#51]
<https://github.com/ocaml-multicore/eio/pull/51>
Sundries
┈┈┈┈┈┈┈┈
• [ocaml-multicore/eventlog-tools#2] Add a pausetimes tool
A `eventlog_pausetimes' tool has been added to `eventlog-tools' that
takes a directory of eventlog files and computes the mean, max pause
times, as well as the distribution up to the 99.9th percentiles. For
example:
┌────
│ ocaml-eventlog-pausetimes /home/engil/dev/ocaml-multicore/trace3/caml-426094-* name
│ {
│ "name": "name",
│ "mean_latency": 718617,
│ "max_latency": 33839379,
│ "distr_latency": [191,250,707,16886,55829,105386,249272,552640,1325621,13312993,26227671]
│ }
└────
• [ocaml-multicore/kcas#9] Backoff with `cpu_relax'
The `Domain.Sync.{critical_section, wait_for}' have now been
replaced with `Domain.Sync.cpu_relax', which matches the
implementation with lockfree.
• [ocaml-multicore/retro-httpaf-bench#10] Add Eio benchmark
The Eio benchmark has now been added to the retro-httpaf-bench
GitHub repository.
• [ocaml-multicore/retro-httpaf-bench#11] Do a recursive checkout in
the CI build
The `build_image.yml' workflow has been updated to perform a
recursive checkout of the submodules for the CI build.
• [domainslib#29] Task stealing with Chase Lev deques
The task-stealing Chase Lev deques for scheduling tasks across
domains is now merged, and shows promising results on machines with
128 CPU cores.
• [ocaml-multicore/multicore-opam#55] Add 0.3.0 release of domainslib
The opam file for `domainslib.0.3.0' has been added to the
multicore-opam repository.
[ocaml-multicore/eventlog-tools#2]
<https://github.com/ocaml-multicore/eventlog-tools/pull/2>
[ocaml-multicore/kcas#9]
<https://github.com/ocaml-multicore/kcas/pull/9>
[ocaml-multicore/retro-httpaf-bench#10]
<https://github.com/ocaml-multicore/retro-httpaf-bench/pull/10>
[ocaml-multicore/retro-httpaf-bench#11]
<https://github.com/ocaml-multicore/retro-httpaf-bench/pull/11>
[domainslib#29] <https://github.com/ocaml-multicore/domainslib/pull/29>
[ocaml-multicore/multicore-opam#55]
<https://github.com/ocaml-multicore/multicore-opam/pull/55>
Benchmarking
╌╌╌╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml-bench/sandmark-nightly#1] Cannot alter comparison input
values
The `Timestamp' and `Variant' fields in the dropdown option in the
`parallel_nightly.ipynb' notebook get reset when recomputing the
whole workbook.
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/e/e029e7d420487a4f3f63893dd4412322c1933787_2_1380x278.png>
• [ocaml-bench/sandmark#230] Build for 4.13.0+trunk with dune.2.8.1
The `ocaml-migrate-parsetree.2.2.0' and `ppxlib.0.22.2' packages are
now available for 4.13.0+trunk, and we are currently porting the
Irmin Layers benchmark in Sandmark from using Irmin 2.4 to 2.6.
• [ocaml-bench/sandmark#231] View results for a set of benchmarks in
the nightly notebooks
A feature request to filter the list of benchmarks when using the
Sandmark Jupyter notebooks.
• [ocaml-bench/sandmark#233] Update pausetimes_multicore to fit with
the latest Multicore changes
The pausetimes are now updated for both the 4.12.0 upstream and
4.12.0 Multicore branches to use the new Common Trace Format
(CTF). The generated graphs for both the sequential and parallel
pausetime results are illustrated below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/9/9aa420116b999df5b89cd106b211673b3d2afb1d_2_1380x458.png>
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/c/cc380d59494290056577f76f977b0ffcb4199e55_2_1380x710.png>
• [ocaml-bench/sandmark#235] Update selected benchmarks as a set for
baseline benchmark
The baseline benchmark for comparison should only be one from the
user selected benchmarks in the Jupyter notebooks.
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/f/fba96b8d43ee766bff262ec5ca0c48e356e8d9cb.png>
• [ocaml-bench/sandmark#236] Implement pausetimes support in
sandmark_nightly
The sequential and parallel pausetimes graph results need to be
implemented in the Sandmark nightly Jupyter notebooks. The results
are similar to the Figures 10 and 12 produced in the [Retrofitting
Parallelism ont OCaml, ICFP 2020 paper].
• [ocaml-bench/sandmark#237] Run sandmark_nightly on a larger machine
The testing of Sandmark nightly sequential and parallel benchmark
runs have been done on a 24-core machine, and we would like to
deploy the same on a 64+ core machine to benefit from the recent
improvements to Domainslib.
• [ocaml-bench/sandmark#241] Switch to default Random module
An on-going discussion on whether to switch to using `Random.State'
for the sequential Minilight, global roots micro-benchmarks and
Evolutionary Algorithm.
[ocaml-bench/sandmark-nightly#1]
<https://github.com/ocaml-bench/sandmark-nightly/issues/1>
[ocaml-bench/sandmark#230]
<https://github.com/ocaml-bench/sandmark/pull/230>
[ocaml-bench/sandmark#231]
<https://github.com/ocaml-bench/sandmark/issues/231>
[ocaml-bench/sandmark#233]
<https://github.com/ocaml-bench/sandmark/pull/233>
[ocaml-bench/sandmark#235]
<https://github.com/ocaml-bench/sandmark/issues/235>
[ocaml-bench/sandmark#236]
<https://github.com/ocaml-bench/sandmark/issues/236>
[Retrofitting Parallelism ont OCaml, ICFP 2020 paper]
<https://arxiv.org/pdf/2004.11663.pdf>
[ocaml-bench/sandmark#237]
<https://github.com/ocaml-bench/sandmark/issues/237>
[ocaml-bench/sandmark#241]
<https://github.com/ocaml-bench/sandmark/pull/241>
Completed
┄┄┄┄┄┄┄┄┄
• [ocaml-bench/sandmark#232] `num_domains' -> `num_additional_domains'
The benchmarks have been updated to now use
`num_additional_domains', to be consistent with the naming in
Domainslib.
• [ocaml-bench/sandmark#239] Port grammatrix to Task pool
The Multicore Grammatrix benchmark has now been ported to use
Domainslib Task pool. The time and speedup graphs are given below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/d/d1e2d18707d543bc73cfa967c7dbfc2dd4783416.png>
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/4/4b920039c0ebf9b1fbac4597b08ecf40f6e0f14f.png>
[ocaml-bench/sandmark#232]
<https://github.com/ocaml-bench/sandmark/pull/232>
[ocaml-bench/sandmark#239]
<https://github.com/ocaml-bench/sandmark/pull/239>
OCaml
╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml/ocaml#10039] Safepoints
The PR is currently being testing and evaluated for both ARM64 and
PowerPC architectures, in particular, the branch relaxations applied
to `Ipoll' instructions.
Our thanks to all the OCaml users, developers and contributors in the
community for their continued support to the project. Stay safe!
[ocaml/ocaml#10039] <https://github.com/ocaml/ocaml/pull/10039>
memprof-limits (first official release): Memory limits, allocation limits, and thread cancellation, with interrupt-safe resources
═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-memprof-limits-first-official-release-memory-limits-allocation-limits-and-thread-cancellation-with-interrupt-safe-resources/8135/1>
Guillaume Munch-Maccagnoni announced
────────────────────────────────────
Dear OCamlers, I am satisfied to announce the first official release
of the package `memprof-limits' (v0.2.0).
From the [user guide] (which has detailed examples and use-cases):
This package lets you interrupt *tasks* in a thread-safe
and resource-safe way, when a resource limit is reached or
a cancellation token is set. A task is an isolated piece
of computation running within a thread.
*Global memory limits* interrupt a task when the major
heap exceeds a certain size. *Allocation limits* interrupt
a task when a certain number of words have been allocated,
a portable mesure of time elapsed and quantity of work
done. *Token limits* interrupt an allocating task when an
arbitrary token is set.
It is available on opam for OCaml ≥ 4.12.
The name comes from the fact that it uses OCaml's `Memprof'
statistical profiler engine (to perform limit checks
frequently-enough, related to the allocation rate of the program, and
at the same time rarely-enough to not affect performance).
It comes with an implementation of *masking* that lets you define
interrupt-safe resources that are guaranteed to be cleaned-up, and
with them [ensure that your program remains in a valid state after
being interrupted].
To learn more about it, I recommend again the [user guide].
[user guide]
<https://guillaume.munch.name/software/ocaml/memprof-limits/>
[ensure that your program remains in a valid state after being
interrupted]
<https://guillaume.munch.name/software/ocaml/memprof-limits/recovering.html>
Example
╌╌╌╌╌╌╌
You can try it with OCaml 4.12 on the following example (`opam install
memprof-limits' -> `utop' -> `#require "memprof-limits";;'):
A worker task allocates 300M words, only 3k of which live
simultaneously. This comes close to an allocation limit of 330M words
set by the monitor. The probability that the worker is interrupted is
less than 10^-50, and thus the computation successfully completes, in
about a second.
┌────
│ (* worker *)
│ let f () =
│ let rec alloc n x =
│ if n = 0 then x else alloc (n-1) (()::x)
│ in
│ (* allocate 300'000 kw *)
│ for i = 0 to 100_000 do ignore (alloc 1_000 []) done
│
│ (* monitor *)
│ let g () =
│ match Memprof_limits.limit_allocations ~limit:330_000L f with
│ | Ok ((), n) -> Printf.printf "success (est. alloc. %#Lu kw)\n" n
│ | Error _ -> print_endline "out of fuel"
│
│ (* main *)
│ let () =
│ Memprof_limits.start_memprof_limits () ;
│ g ()
└────
Changes
╌╌╌╌╌╌╌
Major changes to last year's experimental release:
• The internal state is now protected against asynchronous exceptions
arising from memprof callbacks (especially the own interrupt of
memprof-limits), so it is no longer buggy (“experimental”).
• And now it works under bytecode too.
• Added token limits: interrupts that can be triggered at a distance.
• Detailed guide and reference manuals, now generated by `odoc'.
• [Features to program with interrupt- and resource-safety in mind].
• API revamp.
[Features to program with interrupt- and resource-safety in mind]
<https://guillaume.munch.name/software/ocaml/memprof-limits/index.html#isolation>
Links
╌╌╌╌╌
• [Gitlab repository].
• [Guide with use cases and examples].
• [Reference manual].
• A [statistical analysis of the different limits].
• A [guide to recover from interrupts].
• [FAQ].
[Gitlab repository] <https://gitlab.com/gadmm/memprof-limits/>
[Guide with use cases and examples]
<https://guillaume.munch.name/software/ocaml/memprof-limits/>
[Reference manual]
<https://guillaume.munch.name/software/ocaml/memprof-limits/Memprof_limits/>
[statistical analysis of the different limits]
<https://gitlab.com/gadmm/memprof-limits/-/blob/master/doc/statistical.md>
[guide to recover from interrupts]
<https://guillaume.munch.name/software/ocaml/memprof-limits/recovering.html>
[FAQ] <https://gitlab.com/gadmm/memprof-limits/-/blob/master/FAQ.md>
Bitwuzla 1.0.0 (SMT solver for AUFBVFP)
═══════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-bitwuzla-1-0-0-smt-solver-for-aufbvfp/8138/1>
Frédéric Recoules announced
───────────────────────────
On behalf of the bitwuzla team I am pleased to announce the release of
bitwuzla ocaml binding (<https://github.com/bitwuzla/ocaml-bitwuzla>).
Bitwuzla (<https://bitwuzla.github.io/>) is a Satisfiability Modulo
Theories (SMT) solver for the theories of fixed-size bit-vectors,
floating-point arithmetic, arrays and uninterpreted functions and
their combinations.
This ocaml binding comes in two flavors:
• a straight low-level C binding `Bitwuzla_c' (for the brave and the
unaware) – `opam depext -i bitwuzla-c';
• a type-safier OCaml API `Bitwuzla' – `opam depext -i bitwuzla'.
(Additionally, you can build bitwuzla standalone executable with `opam
depext -i bitwuzla-bin'.)
Documentation and examples are available online
(<https://bitwuzla.github.io/docs/ocaml/>).
Feel free to give it a try,
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <https://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <https://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20210713/244202cc/attachment-0001.html>
More information about the caml-news-weekly
mailing list