[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Jul 25 01:45:47 PDT 2023
Hello
Here is the latest OCaml Weekly News, for the week of July 18 to 25,
2023.
Table of Contents
─────────────────
Ppxlib dev meetings
Ocamls spotted in the wild
DkML 2.0.0 Preview Release
Next priority for OCaml?
Explorations on Package Management in Dune
OCamlFormat 0.26.0
Tarides Office Hours: Benchmark Tooling
You Can Attend the New OCaml.org Community Meetings
euler, an arithmetic library for native integers
DkML Install API 0.4.0
Liquidsoap 2.2.0 is out! 🎉
kcas and kcas_data 0.6.1: STM and compositional lock-dree data structures
ctypes 0.21.1, now with dune support
Announcing the ocaml-wasm organisation
Feedback Needed: New “Arrays” Tutorial on OCaml.org
Old CWN
Ppxlib dev meetings
═══════════════════
Archive: <https://discuss.ocaml.org/t/ppxlib-dev-meetings/12441/5>
Continuing this thread, Sonja Heinze announced
──────────────────────────────────────────────
Here are the meeting notes of the July meeting:
<https://github.com/ocaml-ppx/ppxlib/wiki/Dev-meeting-18-07-2023>
As a heads-up: There won’t be a meeting in August, so the next one
will be in September. Have a nice summer/winter/other season everyone!
:)
Ocamls spotted in the wild
══════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocamls-spotted-in-the-wild/12641/1>
Thorsten Ohl announced
──────────────────────
Here’s a paper on algorithms for theoretical particle physics
referring to ocaml published today:
[Functional directed acyclical graphs for scattering amplitudes in
perturbation theory]
[Functional directed acyclical graphs for scattering amplitudes in
perturbation theory]
<https://link.springer.com/article/10.1140/epjc/s10052-023-11787-2>
gasche then added
─────────────────
It looks like this paper builds on previous work implementing a tool
called Omega in OCaml, which has been around since 2001:
O’Mega: An Optimizing Matrix Element Generator
Mauro Moretti, Thorsten Ohl, Jürgen Reuter, 2001
<https://arxiv.org/abs/hep-ph/0102195>
DkML 2.0.0 Preview Release
══════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-dkml-2-0-0-preview-release/12642/1>
jbeckford announced
───────────────────
There are several major changes in version 2.0.0 of DkML, a OCaml
distribution that uses Visual Studio on Windows. Full DkML 2.0.0
Release Notes are available, and the installer is
<https://gitlab.com/diskuv-ocaml/distributions/dkml/-/releases/2.0.0/downloads/setup64u.exe>.
Consider 2.0.0 a preview release. If you are risk-averse you may want
to wait for 2.0.1.
DkML 2.0.0 Release Notes:
<https://gitlab.com/diskuv-ocaml/distributions/dkml/-/releases#2.0.0>
*Breaking change*: The global environment (where you can run `dune',
`utop', `ocamlopt' and a few other critical OCaml executables without
having to install your own opam switch) has changed significantly.
[DkML 2.0.0 Release Notes] describes those changes in detail.
Why do this breaking change?
1. 2.0.0+ installations are faster because the global executables do
not have to be compiled.
2. It gets the DkML distribution very close to having a lite “bytecode
only” installer that does not need to install (heavy!) Visual
Studio, MSYS2 and Git. /This upcoming lite installer could be a
good fit for educational settings, so feedback from educators is
appreciated./
Major non-breaking changes:
• The deprecated `fdopen' repository is no longer used. Previously
DkML installed a smaller and smaller portion of the `fdopen'
repository with each subsequent version. Now DkML 2.x.x relies only
on the central opam repository and the DkML opam repository.
• Dune upgraded to 3.8.3. Among other things, the installer no longer
installs a C# file watcher proxy for `dune build -w'; instead Dune
uses its own native Windows file watcher.
• The set of pinned packages during a `dkml init' local switch has
gone down from approximately 4000 to 200. Rather than pinning each
package in the opam universe to a heuristically-determined version,
we pin only packages that we successfully build on Windows.
• The following packages are accessible by just typing `utop' without
installing a switch:
‣ [`refl'] reflection package
‣ [`graphics'] package
‣ [`base'] package
Bug fixes:
• The installer can now restart after a failed installation, without
having to use the uninstaller.
• Allow setup to succeed on Windows filesystems that do not support
the setting of file attributes
• FlexDLL object files `flexdll_initer_msvc64.obj' and
`flexdll_msvc64.obj' (or 32-bit variant on 32-bit installs) are
installed alongside `flexlink.exe' so flexlink can be used by itself
without setting `FLEXDIR' environment variable.
Changed packages:
• Many packages are upgraded. Here is a sampling of those upgrades:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Package From To
────────────────────────────────────────────────
dune 3.6.2 3.8.3
utop 2.10.0 2.13.1
ptime 0.8.6-msvcsupport 1.1.0
flexdll 0.42 0.43
base v0.15.1 v0.16.1
yojson 2.0.2 2.1.0
stdcompat 19+optautoconf
metapp 0.4.4+win
ocamlformat 0.24.1 0.25.1
lsp 1.12.2 1.16.2
cmdliner 1.1.1 1.2.0
alcotest 1.6.0 1.7.0
ocamlfind 1.9.1 1.9.5
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[`refl'] <https://github.com/thierry-martinez/refl#readme>
[`graphics'] <https://github.com/ocaml/graphics#readme>
[`base'] <https://github.com/janestreet/base#readme>
later on, jbeckford added
─────────────────────────
DkML 2.0.1
╌╌╌╌╌╌╌╌╌╌
Version 2.0.1 is available with an installer and release notes at
<https://gitlab.com/diskuv-ocaml/distributions/dkml/-/releases/2.0.1>
Its main new feature is that the OCaml package `sqlite3' is available
from the global environment. Examples of `sqlite3' are in the [Quick
Start].
If you installed 2.0.0 you should upgrade to 2.0.1 for the bug fixes.
[Quick Start]
<https://gitlab.com/diskuv-ocaml/distributions/dkml#quick-start>
Next priority for OCaml?
════════════════════════
Archive:
<https://discuss.ocaml.org/t/next-priority-for-ocaml/12561/62>
Deep in this thread, gasche said
────────────────────────────────
I would be hard pressed to mention /one/ priority for OCaml, because I
think that many things are important – and important in different ways
for different people. I thought that I could give some information on
what support I have been involved in, either as a compiler maintainer
or as a member of the [OCaml Software Foundation], on some of the
topics that have been mentioned here.
• Modular implicits: we’ve been trying to find people we could fund to
make progress on this, but it is difficult – it requires a lot of
type-system expertise, and time. We are funding internships with
@yallop in Cambridge to experiment with the system, and I am
thinking of getting my hands dirty during the next school year with
some module-side implementation work – possibly diving into the PRs
of Matthew Ryan.
• Type system for effects: I have decided to setup an informal working
group for people interested in working on this (mostly academics,
also Jane Street) to exchange information, and in particular to
decide a common set of evaluation criteria – with compatibility with
existing non-effect-using OCaml code as a top priority. I hope to
announce something more structured next September, with the aim of
getting a clearer idea of potential design proposals by the end of
the academic years.
• Debuggers: the OCaml Foundation funded ocamlearlybird’s author in
2020-2021, hackwaly, before they stopped to work full-time for a
startup.
(<https://discuss.ocaml.org/t/ann-ocamlearlybird-now-an-ocaml-software-foundation-supported-project/6834>
, <https://discuss.ocaml.org/t/ann-ocamlearlybird-1-0-0-beta1/7180>
, etc. ) I learned a week ago that @sim642 is interested in
continuing maintenance of the project, and we are looking at funding
his work. (ocamlearlybird is a bytecode debugger, similar to the
venerable `ocamldebug' – it is a client for the same debugger
protocol. Separately we would ideally have good debugging support
with the native compiler. That aspect of the debugging story has
been worked on by Mark Shinwell in the past, but there is a
disagreement among compiler maintainers about what is the right
technical approach to support DWARF information in the compiler – it
is a complex format that is not necessarily a good match for OCaml,
and adding it to the compiler backend is fairly invasive.)
• A priority that people don’t know about, but in my mind comes before
“typed effects”, is to refactor and cleanup the implementation of
the OCaml type checker, which has historically been a place of
technical debt in the compiler. There has been a fair amount of work
on this in the last year, driven by Jacques Garrigue as his group at
Nagoya ( in particular Takafumi Saikawa ), with Florian Angeletti
and myself on the reviewer side, and in the last few months some
very welcome help from the Jane Street group working on language
features (in particular Richard Eisenberg, Chris Casinghino, Nick
Roberts) who have been lending a hand on refactoring and
refactoring-reviewing work. More generally, I have been worried this
year about maintenance workforce for the OCaml compiler codebase
(<https://discuss.ocaml.org/t/maintenance-bottlenecks-in-the-compiler-distribution/11045>
). I think that solving this issue is also a priority, at the level
of the compiler distribution. We have made some progress, with
notably maintenance contributions from OCamlPro and Jane Street, but
I think that the situation still requires careful monitoring.
• Relocatability: I agree with @smorimoto that this is important. I
would want all OCaml packages to be relocatable, notably as the
right first step to enable caching or even binary distribution of
OCaml package build artifacts. There has been work in this direction
from David Allsopp in the last few years, which I understand gets
closer every year to being in a state that could be submitted as
upstream PRs. My strategy so far is to wait for this to make
progress.
• Tooling, things that directly come to mind are:
⁃ development-environment tools whose recommended workflow is to
have separate/isolated development environment(s) for each
project. The `opam' client allows this with local switches, but I
think the UI could be streamlined a bit to make this the easy
default. (For example, not everyone knows how to build a local
switch with only the dependencies of the project installed, to
start hacking on it.) Ideally we would have caching of package
builds across those development environments, and maybe even some
distributed caching of build artifacts (assuming relocatability).
⁃ a “code upgrade” tool to which I can specify program
transformations (for example: “rewrite `SomeLib.somefun $bar $baz'
into `SomeLib.somefun $bar (Somelib.convert $baz)'”, and it
applies it to my source code in a diff-friendly way.
[OCaml Software Foundation] <https://ocaml-sf.org/>
Explorations on Package Management in Dune
══════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/explorations-on-package-management-in-dune/12101/25>
Deep in this thread, Thomas Gazagnaire said
───────────────────────────────────────────
My view on this. When people talk about Opam, this covers different
aspects.
• the opam files and files repositories. The main repository is (for
us OCamlers) `ocaml/opam-repository', but the [Coq community also
has
one](<https://github.com/ocaml/opam/wiki/Dev-Meeting-2020.11.06-(opam-and-coq-repos)>),
and many companies use their private repository. We do not want to
break this workflow. The Dune package management proposals aim to
stay fully compatible with this workflow and work with any opam
packages (using `dune' or not). There’s also exciting ongoing work
for [package
signing](<https://github.com/ocaml/opam/wiki/Dev-Meeting-2021.06.11-(Conex)>)
that we do want to see land at one point. We have also built and are
operating an extensive CI infrastructure around these repositories –
for instance, [ocaml-repo-ci] is building 100,000 jobs daily on all
the Tier1 supported platforms for OCaml. We do not want to rebuild
this once more time!
• the opam client(s). The main one is the `opam' CLI, but many more
tools use the opam files metadata. There are a [few
tentatives](<https://github.com/ocaml/opam/wiki/Dev-Meeting-2020.10.23-(opam2nix)>)
of generating `nix' derivations for those files. There’s also
[esy](<https://github.com/ocaml/opam/wiki/Dev-Meeting-2020.12.04-(esy-drom)>)
and the package managers that try to close the gap between the OCaml
and JavaScript ecosystems. The client is built around a library
(`opam-lib'), but this has never been designed properly. When I
wrote opam initially, it was only focused on the CLI. Later with
@AltGr, we tried to split it a bit more cleanly, but the API is
still painful to use (for instance, every function that needs to
load the filesystem needs a value that holds that state that would
take dozens of arguments - then, as these functions perform
file-system or network effects, you somehow need to keep these
values synchronised with the new filesystem state – that’s painful
and error prone). This API can somehow be split into various parts:
1. Reading the opam repository state: parsing opam files, building a
dependency graph (that’s the part you mention @gasche)
2. Resolving constraints: opam has a pluggable interface for
constraint solving, and by default, it will use what solver is
installed on your system (or some internal heuristics which used
to be very naive but seems much better nowadays). Opam needs to
serialise and parse solver requests, including solver errors that
need to be somehow pretty printed to the user.
3. If the solver can devise a building plan, parse it and prepare it
by downloading (and caching) the build/package sources.
4. run the build commands for all the packages and install them
locally.
Nowadays, most package managers also have the option to snapshot the
state of the build plan between 2. and 3. That’s for instance, what
tools like `opam-monorepo lock' or `opam lock' are doing. But when you
do `opam install --locked', `opam' still calls the solver (to check
that your lock file is consistent and complete), so doing 3-2-3-4. And
`opam-monorepo pull' does 3. and delegate 4. to `dune build' (so all
your dependencies need to use `dune' and be co-installable in a `dune'
workspace).
So to come back to your question @gasche: The Dune package management
experiment will be using `opam-lib' to do 1, 2 and 3 (whether it’s the
current `opam-lib' or an improved version that relies a bit less on
filesystem state is still in discussions - whatever the result is it
will be upstreamed). And it will be using the `dune' scheduler to do 4
(but still using the opam build instructions).
[ocaml-repo-ci] <https://github.com/ocurrent/opam-repo-ci>
OCamlFormat 0.26.0
══════════════════
Archive: <https://discuss.ocaml.org/t/ann-ocamlformat-0-26-0/12649/1>
Jules announced
───────────────
We are happy to announce the release of OCamlFormat 0.26.0, the
auto-formatter for OCaml code.
This release contains changes that might affect your code and overall
projects. It could include impact to module arguments, `fun' passed as
a labelled argument, and indentation within parentheses.
The complete [list of changes is here]. Items marked with a `*' are
likely to change your code.
The OCamlFormat dev team
[list of changes is here]
<https://github.com/ocaml-ppx/ocamlformat/releases/tag/0.26.0>
Tarides Office Hours: Benchmark Tooling
═══════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-tarides-office-hours-benchmark-tooling/12655/1>
Shakthi Kannan announced
────────────────────────
The [current-bench] infrastructure and pipeline are used for
benchmarking the OCaml compiler and ecosystem projects. It uses the
[OCurrent] pipeline to run predictable, I/O-bound benchmarks for the
respective OCaml projects.
We have successfully added benchmarks for Dune, Lockfree, odoc,
Merlin, Eio, opam, etc., projects at [https://autumn.ocamllabs.io]. We
would now like to make our infrastructure available to the larger
community to increase adoption and continue our work on performance
analysis, tooling, and benchmarking.
You can follow the instructions in the current-bench [README] to
create a benchmark for your project and install the GitHub application
to onboard your OCaml project to current-bench.
The Tarides Benchmark Tooling team is also making open office hours
available for you to engage, ask questions, and work with us on
benchmarking OCaml projects. We will be available online, every
Monday, between 1030 and 1130 CET on Google Meet as mentioned below:
Tarides Office Hours: OCaml Benchmarking Starting Monday, July 24,
2023 (1030 - 1130 CET) (1400 – 1500 IST)
Google Meet joining info: Video call link:
<https://meet.google.com/rcu-wjpk-jyh> Or dial: (US) +1 440-549-4808
PIN: 584 331 902#
More phone numbers: <https://tel.meet/rcu-wjpk-jyh?pin=4479584706118>
Please feel free to join our weekly Monday sessions, and we will be
happy to interact with the community and enhance our benchmarking
efforts for OCaml!
[current-bench] <https://github.com/ocurrent/current-bench>
[OCurrent] <https://github.com/ocurrent/ocurrent>
[https://autumn.ocamllabs.io] <https://autumn.ocamllabs.io>
[README]
<https://github.com/ocurrent/current-bench/blob/main/README.md#enroll-your-repository>
You Can Attend the New OCaml.org Community Meetings
═══════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/you-can-attend-the-new-ocaml-org-community-meetings/12656/1>
Sabine Schmaltz announced
─────────────────────────
the OCaml.org maintainers are now holding community meetings, on
Mondays every two weeks on Zoom.
Here are some dates for the upcoming meetings [(scheduled on Google
Calendar)]:
• 24 Jul 2023, Mon 18:00 – 19:00 CEST
• 7 Aug 2023, Mon 18:00 – 19:00 CEST
• 21 Aug 2023, Mon 18:00 – 19:00 CEST
• 4 Sept 2023, Mon 18:00 – 19:00 CEST
Topics may include:
• Proposals you want to champion
• Q&A sessions about the project
• Guidance on contributing
But we’re not limiting topics to just these. Many more topics relevant
to OCaml.org and how to best make use of it for the benefit of the
OCaml community can also be discussed. If you’d like to add a
topic/issue to the discussion for a specific meeting, please contact
Sabine on [OCaml Discuss] or the [OCaml.org Discord channel], or [open
an issue on the ocaml/ocaml.org GitHub Repository]
P.S.: <https://ocaml.org/news/invitation-to-contribute>
[(scheduled on Google Calendar)]
<https://calendar.google.com/calendar/u/0?cid=Y18wMmU0MWIyYWQwMDE2NWI0MDQ3ODlmYWQ3YzVkZjdlYTNhNGFlNjI3NDRjMTMzMDdlZTdmYjNmZDczODJmMWRkQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20>
[OCaml Discuss] <https://discuss.ocaml.org/u/sabine>
[OCaml.org Discord channel]
<https://discord.com/channels/436568060288172042/1126433906976112700>
[open an issue on the ocaml/ocaml.org GitHub Repository]
<https://github.com/ocaml/ocaml.org/issues>
euler, an arithmetic library for native integers
════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-euler-an-arithmetic-library-for-native-integers/12482/10>
Continuing this thread, glen announced
──────────────────────────────────────
Thanks to @mseri, `euler' is now on opam! And in the meantime it has
gained more functions, the most striking of which being
[`Primes.prime_seq : int -> int Seq.t'] (thank @cuihtlauac for
pointing me to the algorithm!) and [`Arith.smallest_root : int -> int
* int'] (the latter in the next release, 3.0, not on opam yet).
Also, since I still cannot edit the first post, this is a reminder
that the project has moved to GitHub: *[repo]*, *[doc]*. The links in
my first message won’t be updated.
I wonder if this is a useful addition to [Owl ], the ocaml
“fat” math library.
Perhaps? I am not very familiar with Owl (ask Owl’s Benevolent
Dictators For Life? ;-) ). Euler is an integer arithmetic library
whereas, as I understand it, Owl is mainly a floating-point math
library for scientific computing (linear algebra, differentiation,
statistics and and such). I would guess that prime numbers and friends
are rather remote concerns for Owl users, but perhaps Owl is
interested in expanding its domains? I did spot [a small number of
integer arithmetic functions in Owl], most of which are also found in
Euler.
[`Primes.prime_seq : int -> int Seq.t']
<https://gmevel.github.io/euler-lib/index.html/euler/Euler/Primes/index.html#val-prime_seq>
[`Arith.smallest_root : int -> int * int']
<https://gmevel.github.io/euler-lib/index.html/euler/Euler/Arith/index.html#val-smallest_root>
[repo] <https://github.com/gmevel/euler-lib>
[doc] <https://gmevel.github.io/euler-lib/index.html/euler/Euler/>
[Owl ] <https://github.com/owlbarn/owl>
[a small number of integer arithmetic functions in Owl]
<https://github.com/owlbarn/owl/blob/48434ea/src/owl/maths/owl_maths.mli#L587-L607>
DkML Install API 0.4.0
══════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-dkml-install-api-0-4-0/12665/1>
jbeckford announced
───────────────────
I’m pleased to announce that DkML Install API 0.4.0 is available on
opam. This library is what I’ve been using to create the DkML Windows
distribution installer.
TLDR: To create your own installer for your own software, you define
“components” which have dependencies on other components. A component
is an OCaml module you create with four essential functions:
`admin_install' and `admin_uninstall' for escalated privileges, and
`user_install' and `user_uninstall' for normal user privileges. Each
component almost always needs to access assets which can be defined in
the `install:[]' section of an `.opam' file. The net effect is you can
use opam and OCaml to define your own reproducible installer.
There is a walk-through at
[https://diskuv.github.io/dkml-install-api/doc/packages/Console.html]
and more general documentation at
[https://github.com/diskuv/dkml-install-api#readme]
The library works well for the DkML Windows distribution, although it
is too complicated to recommend for casual use today. Regardless, if
someone has a burning desire to create their own installer, and that
installer has reasonably complex logic that needs to run on the
end-user machine, then take a look at DkML Install API! /Expect to see
a steady but slow series of changes to the API until it hits 1.0.0./
Limitations: Today it can make an end-to-end installer for Windows. I
haven’t needed yet to create macOS and Linux installers, although 95%
of the pieces are present. If you are interested in creating the final
packaging step for those OS-es (ex. a `.deb' apt packager for Debian),
feel free to contribute a PR!
[https://diskuv.github.io/dkml-install-api/doc/packages/Console.html]
<https://diskuv.github.io/dkml-install-api/doc/packages/Console.html>
[https://github.com/diskuv/dkml-install-api#readme]
<https://github.com/diskuv/dkml-install-api#readme>
Liquidsoap 2.2.0 is out! 🎉
═══════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-liquidsoap-2-2-0-is-out/12667/1>
Romain Beauxis announced
────────────────────────
We are delighted to announce the release of **Liquidsoap 2.2.0**! It
is now available on [our release page] and should be available [via
opam] shortly!
Liquidsoap is a statically typed scripting general-purpose language
with dedicated operators and backend for all thing media, streaming,
file generation, automation, HTTP backend and more.
This release comes almost 4 months after the last stable release from
the `2.1.x' release cycle and 14 months (!) after forking the `2.2.x'
release branch! It includes some exciting changes in track
muxing/demuxing, HLS, sound processing and more. See below for a
detailed list!
[our release page]
<https://github.com/savonet/liquidsoap/releases/tag/v2.2.0>
[via opam] <https://opam.ocaml.org/packages/liquidsoap>
✨ New features
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Here are the highlights:
◊ 🎛️ Multitrack
This is by far the biggest change in this relase! This brings the
ability to demux and remux tracks inside sources, making it possible
to:
• Encode video with multiple audio tracks
• Create audio-only and audio/video streams from the same content, for
instance a regular audio stream and one with the studio’s video.
• Drop or specify which metadata or track marks track should be used.
• Apply specific audio effect or encoding to different tracks
And much more! The full documentation is [here]. We plan on expanding
this support in the future in particular to allow track selection
based on language, encoded content etc.
[here] <https://www.liquidsoap.info/doc-2.2.0/multitrack.html>
◊ 🏷️ HLS metadata
At last! We now support metadata in HLS streams using a timed ID3
stream for `mpegts' container and plain ID3v2 tags for `adts', `mp3',
`ac3' and `eac3' formats. There is currently no support for metadata
with `mp4' containers.
This feature is *enabled by default* so you might want to check if it
impacts your listeners before pushing it to production. It can be
turned off by setting `id3' to false on your HLS streams.
Full documentation is [here].
[here]
<https://www.liquidsoap.info/doc-2.2.0/hls_output.html#metadata>
◊ 🎨 Colored logs
Small change but not the least important! Logs are now colored by
default when printed on the console. This finally makes it possible to
read logs with high level of details!
We are aware of the need for more developer tooling and quality of
life improvements! In the next release cycle, we hope to work on code
formatting, highlighting and more!
◊ 🕸️ New HTTP API
Interacting with your scripts is essential and, for this, web
interfaces and APIs are really useful. In order to make our HTTP
server easier to use, we wrote a new web API that is very close to
node express’ API and should be fairly easy to use! The documentation
is [here]
These changes also included a revamping of our SSL support which is
now modular and with a new TLS optional support!
[here]
<https://www.liquidsoap.info/doc-2.2.0/harbor_http.html#nodeexpress-api>
◊ 🎚️ Native stereotool support
While commited to open-source through and through, we also do want to
meet our users where they are. To this end, it seems that a lof of
them want to use the proprietary stereotool audio processing. Up until
this version, the only option was via the external command line
encoder and this was not satisfactory.
With this release, it is now possible to use the shared library
distributed by the author, which provides support for an new
`stereotool' internal operator that is much easier to integrate. See
the documentation [here].
[here] <https://www.liquidsoap.info/doc-2.2.0/stereotool.html>
◊ 📟 Records enhancements
As part of the language changes requred for multitrack, we now support
the following operations on records:
Record spread:
┌────
│ let {foo, bar, ...rest} = someRecord
│
│ let otherRecord = { bla = "blo", ...someRecord }
└────
Additionally, we now support optional record methods, for instance:
┌────
│ def f(x) =
│ if x?.foo == null() then
│ print("x does not have method foo")
│ else
│ print("x has method foo")
│ end
│ end
└────
◊ 🪢 Support for YAML parsing/rendering
Following the recent [JSON parsing] feature, we now support [YAML
parsing] in a very similar way as json.
[JSON parsing] <https://www.liquidsoap.info/doc-2.2.0/json.html>
[YAML parsing] <https://www.liquidsoap.info/doc-2.2.0/yaml.html>
◊ 🔮 Memory optimization
While we are aware that memory consumption with this release may have
increased a bit due to on-going changes, we have done our best to
introduce more ways to control it and understand its usage.
In particular, we now support the alternative [jemalloc]() memory
allocator, enabled in all our release assets and configurable via the
internal settings.
We also introduced two new audio content formats, `pcm_s16' and
`pcm_f32' that can be used to store audio internally as, resp., 16-bit
signed integers or 32-bit floating point numbers. Our default internal
format being OCaml’s native 64-bit floating point numbers.
We also added a new `track.audio.defer' operator that can be used to
buffer large amount of audio data without impacting performances.
You can read more about memory utilization in liquidsoap [here].
[here] <https://www.liquidsoap.info/doc-2.2.0/memory.html>
◊ 🐪 Switch to `dune' and javascript runtime
While perhaps more exciting to developers, the project has now fully
moved to the OCaml `dune' builder. This provides an extra level of
flexibility, In particular, we were able to extract the code that is
specific to the liquidsoap language, that is everything that pertains
to parsing/evaluating/type checking without the streaming and system
specific operators and export it as an [online playground]. We’re not
sure yet what we’ll do with it. It might be possible, for instance, to
write a javascript backend to use liquidsoap scripts with the [Web
media APIs]!
[online playground] <https://www.liquidsoap.info/try/>
[Web media APIs] <https://developer.mozilla.org/en-US/docs/Web/Media>
🕵️ Versioning and release assets
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
For a little over a year now, we have switched to _rolling release
cycles_ with maintenance and bugfixes applying only to the current
release cycle. Regular releases are tagged `vX.Y.Z' (for instance
`v2.2.0') on github and docker while on-going releases are tagged
`rolling-release-vX.Y.Z'.
When an initial release, for instance `2.2.0', is being worked on,
bugfixes and user issues were being addressed for users using the
`2.1.x' releases. While we would like to extend support, this is the
best that we can do with our limited resources!
At any given time, the `rolling-release-vX.Y.Z' denotes the released
being worked on. For stable releases branches, this would be for
instance, `rolling-release-v2.2.1' after release `v2.2.0'. For a
yet-to-be released initial version, this would be for instance
`rolling-release-v2.3.x'. We try our best to make those releases as
bug-free as possible. Using one of them to test your production script
also guarantees the fastest response to bugs and issues!
For release assets, we try to maintain two packages for debian and
ubuntu distributions, one with the latest LTS or stable and one with a
recent release. The new `minimal' assets are, as the name suggests,
_minimal_ builds. They contain a limited set of features and standard
library operators. Minimal builds are intended for most production run
and should limit the risk for issues such as segfault and etc. If your
script can run with it, we recommend it over the fully featured
builds.
For each release asset, you can consult the associated `.config' file
to see which features are enabled.
Docker release images are located at: `savonet/liquidsoap:v2.2.0'. The
release tag may be updated if needed. You can use git sha-based images
to pick a fixed build, e.g. `savonet/liquidsoap:<sha>'
Lastly, we may update the list of release assets on the github release
page. If you are looking for permanent release asset links make sure
to checkout [savonet/liquidsoap-release-assets].
[savonet/liquidsoap-release-assets]
<https://github.com/savonet/liquidsoap-release-assets>
🧮 Migration guide
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
We listed most of the migration issues you might run into on [this
page]. The detailed changelog below may also help.
As a reminder, we strongly recommend to test your script in a stagging
environment, even between minor releases, to make sure that everything
is working correctly before pushing a new liquidsoap version to
production!
[this page] <https://www.liquidsoap.info/doc-2.2.0/migrating.html>
kcas and kcas_data 0.6.1: STM and compositional lock-dree data structures
═════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-kcas-and-kcas-data-0-6-1-stm-and-compositional-lock-dree-data-structures/12674/1>
Vesa Karvonen announced
───────────────────────
I’m happy to announce that version 0.6.1 of `kcas' and `kcas_data' has
been released on opam.
See the [Kcas] project for more details, but briefly [*kcas*] provides
a software transactional memory (STM) implementation and [*kcas_data*]
provides compositional lock-free data structures implemented using
Kcas. Together these libraries allow one to implement safe concurrent
abstractions compositionally.
Aside from internal improvements, notable features added since the
[previously announced 0.3.0] version include:
• Doubly linked list [`Dllist'], see [example].
• Synchronizing variable [`MVar'], see [example].
• [Nested conditional transactions], see [example].
• Timeouts, see [example].
• Support for OCaml 4.14, allowing Kcas to be used to implement
systhread and parallelism safe code and help projects requiring
OCaml 4 support on their way to OCaml 5.
Stay tuned for more news on Kcas!
[Kcas] <https://github.com/ocaml-multicore/kcas/#readme>
[*kcas*]
<https://ocaml-multicore.github.io/kcas/doc/kcas/Kcas/index.html>
[*kcas_data*]
<https://ocaml-multicore.github.io/kcas/doc/kcas_data/Kcas_data/index.html>
[previously announced 0.3.0]
<https://discuss.ocaml.org/t/ann-kcas-and-kcas-data-0-3-0-software-transactional-memory/12085>
[`Dllist']
<https://ocaml-multicore.github.io/kcas/doc/kcas_data/Kcas_data/Dllist/index.html>
[example]
<https://github.com/ocaml-multicore/kcas#a-transactional-lru-cache>
[`MVar']
<https://ocaml-multicore.github.io/kcas/doc/kcas_data/Kcas_data/Mvar/index.html>
[example]
<https://github.com/ocaml-multicore/kcas#the-dining-philosophers-problem>
[Nested conditional transactions]
<https://ocaml-multicore.github.io/kcas/doc/kcas/Kcas/Xt/index.html#nested-transactions>
[example]
<https://github.com/ocaml-multicore/kcas#log-updates-optimistically>
[example] <https://github.com/ocaml-multicore/kcas#timeouts>
ctypes 0.21.1, now with dune support
════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ctypes-0-21-1-now-with-dune-support/12675/1>
yallop announced
────────────────
I’m happy to announce a new release of [ctypes], 0.21.1, available for
installation via OPAM.
ctypes is a library for binding to C libraries using pure OCaml, with
the primary aim of make writing C extensions as straightforward as
possible.
The core of ctypes is a set of combinators for describing the
structure of C types – numeric types, arrays, pointers, structs,
unions and functions. You can use these combinators to describe the
types of the functions that you want to call, then bind directly to
those functions – all without writing any C!
The major change in this release is a switch to the dune build system,
contributed by Anil Madhavapeddy (@avsm) and Etienne Millon
(@emillon). The benefits of this change for users of ctypes are
described in [the pull request implementing the change]; in summary:
With a port to `dune', the ctypes library can be embedded
in larger dune projects simply by including it in the
directory tree of the bigger project.
The [release notes] have further details of the changes in this
release and recent point releases.
[ctypes] <https://github.com/yallop/ocaml-ctypes/>
[the pull request implementing the change]
<https://github.com/yallop/ocaml-ctypes/pull/588>
[release notes]
<https://github.com/yallop/ocaml-ctypes/blob/0.21.1/CHANGES.md>
Announcing the ocaml-wasm organisation
══════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/announcing-the-ocaml-wasm-organisation/12676/1>
KC Sivaramakrishnan announced
─────────────────────────────
We are happy to announce the creation of the [ocaml-wasm] Github
organisation which will work towards [WebAssembly] (Wasm) support for
OCaml. The goal of the organisation is to enable users to build
high-performance and secure applications for the Wasm platform using
the OCaml programming language and the platform tools.
Wasm is a binary instruction format, designed as a portable
compilation target for programming languages, enabling deployment on
the Web for client and server applications. It provides a sandboxed
execution environment, which also makes it attractive for both
blockchains and Cloud deployment. For example, Fastly’s [Compute at Edge
platform] is based on WebAssembly.
[ocaml-wasm] <https://github.com/ocaml-wasm>
[WebAssembly] <https://webassembly.org/>
[Compute at Edge platform]
<https://docs.fastly.com/products/compute-at-edge>
Compilers
╌╌╌╌╌╌╌╌╌
The ocaml-wasm organisation brings together the ongoing efforts at
compiling OCaml to Wasm.
◊ wasocaml
[OCamlPro] has been developing [wasocaml], [an experimental compiler
backend] for OCaml that targets Wasm starting from the flambda
intermediate representation of the OCaml compiler. OCamlPro engineers
have also been contributing to the Wasm GC working group to ensure
that the GC and other extensions in Wasm are amenable to targeting
OCaml.
[OCamlPro] <https://ocamlpro.com/>
[wasocaml] <https://github.com/ocaml-wasm/wasocaml>
[an experimental compiler backend]
<https://ocamlpro.com/blog/2022_12_14_wasm_and_ocaml/>
◊ wasm_of_ocaml
Alongside this effort, [Tarides] has been working on implementing
[wasm_of_ocaml], a toolchain to compile OCaml to WebAssembly based on
the battle-tested [js_of_ocaml] compiler. Both js_of_ocaml and
wasm_of_ocaml use OCaml bytecode as the input and emit JavaScript and
Wasm respectively.
It is notable that these efforts started off jointly a few years ago
with [an exploration] aimed at adding a Wasm backend to OCaml. At that
time, it was clear that Wasm did not have enough features to support
OCaml efficiently. With the implementation of several key Wasm
extensions in browser engines, the time is right for this effort.
There exist other approaches to running OCaml programs using Wasm
runtimes, where the speed of generated code is less of a concern. For
example, a relatively straightforward approach is to compile the OCaml
bytecode interpreter to Wasm and interpret the bytecode programs.
[ocamlrun-wasm] and [wasicaml] are ports of the OCaml bytecode
interpreter to Wasm. Wasicaml furthermore has a compiler mode somewhat
similar to wasm_of_ocaml (but simpler) that parses a bytecode
executable and translates it into Wasm.
[Tarides] <https://tarides.com/>
[wasm_of_ocaml] <https://github.com/ocaml-wasm/wasm_of_ocaml>
[js_of_ocaml] <https://github.com/ocsigen/js_of_ocaml>
[an exploration]
<https://github.com/sabine/ocaml-to-wasm-overview/tree/master>
[ocamlrun-wasm] <https://github.com/sebmarkbage/ocamlrun-wasm>
[wasicaml] <https://github.com/remixlabs/wasicaml>
◊ Evolving Wasm
Wasm is an evolving standard. Wasm 1.0 focuses on low-level support
for compiling low-level languages such as C and C++. There are several
extensions under development which aim to provide support for
targeting high-level languages including [garbage collection], [tail
calls], [stack switching] and [threads]. In particular, the Wasm
development process is completely open and anyone can participate in
the evolution of Wasm.
The OCamlPro engineers [contributed] to the Wasm GC extension with the
help of their experimental flambda to Wasm compiler. Both OCamlPro and
Tarides engineers are actively involved with the Wasm development
process on multiple fronts. Having more than one approach to compile
OCaml to Wasm allows us to build comprehensive evidence and make
informed arguments to influence the design of Wasm extensions. In
addition, a major implementation effort is the runtime system support
for Wasm-compiled OCaml code, which will be shared between the
different OCaml to Wasm compilers.
[garbage collection] <https://github.com/WebAssembly/gc>
[tail calls] <https://v8.dev/blog/wasm-tail-call>
[stack switching] <https://github.com/WebAssembly/stack-switching>
[threads] <https://github.com/WebAssembly/threads>
[contributed] <https://ocamlpro.com/blog/2022_12_14_wasm_and_ocaml/>
OCaml-Wasm monthly update
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
We will post a summary of the work done in the [ocaml-wasm] org on
[OCaml discuss].
/Happy hacking!/
[ocaml-wasm] <https://github.com/ocaml-wasm>
[OCaml discuss] <https://discuss.ocaml.org/>
Feedback Needed: New “Arrays” Tutorial on OCaml.org
═══════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/feedback-needed-new-arrays-tutorial-on-ocaml-org/12683/1>
Sabine Schmaltz announced
─────────────────────────
Hey everyone,
there’s a new tutorial on “Arrays” at
<https://staging.ocaml.org/docs/arrays>
Looking for constructive feedback.
Is there any information missing or questions that you would need
answered in order for this tutorial to be useful? Anything you would
change?
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I’ll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <https://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <https://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20230725/45ed3db8/attachment-0001.html>
More information about the caml-news-weekly
mailing list