[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Nov 24 04:08:20 PST 2020
Hello
Here is the latest OCaml Weekly News, for the week of November 17 to 24,
2020.
Table of Contents
─────────────────
OCaml User Survey 2020
Suggestions from the OCaml Survey result
OCaml for ARM MacOS
TechEmpower benchmark: httpaf + lwt + unix on par with Haskell's warp
Dropbox v2 API, and JSoO bindings for GridJS and IndexedDB
OCaml in Jupyter-based Environments
Ocurrent/opam Docker images have moved to ocaml/opam
ocaml-lsp-server 1.2.0
ca-certs and ca-certs-nss
Compatibility packages for 4.12 (either, semaphore-compat)
Jupyter with pyml
OCaml needs an arbitrary-precision decimal type
Type-at-point ocaml-lsp/merlin in vim/neovim
ocamlearlybird now an OCaml Software Foundation supported project
release of mc2 0.1, a SMT solver
Wednesday, 25th November 2020 is MirageOS Bug Cleaning Day!
Other OCaml News
Old CWN
OCaml User Survey 2020
══════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ocaml-user-survey-2020/6624/25>
Continuing this thread, Xavier Leroy announced
──────────────────────────────────────────────
I extracted the replies to the free-form questions and made them
available as a Gist. There are so many replies that I have no idea
how to exploit and summarize them!
<https://gist.github.com/xavierleroy/88a22b2b8e609c46bf9d23f36521a483>
Patrick Ferris then said
────────────────────────
Thanks for putting together this great resource!
I think it's quite important to take the results from this survey and
find actionable items (or highlight ongoing efforts) which aim to
improve different problems for different users. As a result I have put
together some graphs trying to categorise the answers based on
proficiency (`beginner', `intermediate', `advanced' and
`expert'). These groupings are quite subjective but I thought it might
offer a slightly more nuanced look into what problems users had, where
they come from, what they are doing and what commonalities they
share. The [code is here] and the [HTML version of the notebook here]
(I'm no data-scientist nor python developer :snake: if you see glaring
mistakes do raise an issue!).
Here are some possible inferences (*warning: opinions ahead*) that can
be made from the data along with possible actionable steps (or links
to ongoing projects):
• [Everybody wants better documentation for user libraries]. I think
people want more long-form style documentation. However, excellent
work is currently taking place with [odoc] (see also [OCaml workshop
video]) and perhaps these concerns can be wrapped into this ongoing
work.
• Everybody wants *multicore* but "experts" want *implicits* a little
more – [multicore is coming along amazingly] and [implicits] are in
an earlier but active state AFAICT.
• [A wide variety of application domains]
• less proficient OCaml users tend to be doing more web-related tasks
or data processing. Advanced/expert users are implementing
programming languages, building developer tools, systems and formal
methods. This is reflected in other breakdowns for example in
["other fluent languages"] we see JS more popular with beginners and
C with experts. But also track the progress of *Coq and friends*
moving up as proficiency increases. This is quite interesting and I
think it could point to an actionable item of working more on the
web and formal methods components of the OCaml ecosystem
i.e. working with Coq devs and JS devs to help them.
• From a Javascript point-of-view, looking at the [implementations]
section it's interesting to note across the four proficiency levels
`js_of_ocaml' is more popular than `Reason' except for
beginners. Both projects are actively worked on, but perhaps more
`js_of_ocaml' documentation would be good with many tools already
existing like [gen_js_api], Jane Street's [bonsai] etc. In my
opinion, this survey motivates working on unifying the JS world of
OCaml as (at least to me) right now it feels a little fragmented and
hard for someone new to know where to go.
• [Opam with the public repository] across all users is consistently
the most used installation method – this justifies the large amount
of work that tasks place on [opam] that I'm sure most are very
grateful for. Tying this back to the previous point of "cross
community" work, opam have [community meetings] which seem to be
v. useful.
• Pain points – the top two (although they interchange) is "lack of
critical libraries" and "too hard to find and hire OCaml
developers". Some [effort has been made] to find these libraries
including in the survey. I'm not sure what else can be done. As for
hiring, looking at the different [communication channels] there is
quite a spread with different users using different channels. This
is great, but may also cause some fragmentation and make the
"hiring" process worse.
These are just some thoughts from a relatively new OCaml community
member, I would love to know what other actionable steps people think
we can take from this data and hopefully we can produce a more concise
and specific set of steps to put this survey to great use :))
[code is here]
<https://github.com/patricoferris/ocaml-survey-analysis/blob/main/survey-analysis.ipynb>
[HTML version of the notebook here]
<https://patricoferris.github.io/ocaml-survey-analysis/>
[Everybody wants better documentation for user libraries]
<https://patricoferris.github.io/ocaml-survey-analysis/#State-of-the-Art->
[odoc] <https://github.com/ocaml/odoc>
[OCaml workshop video] <https://www.youtube.com/watch?v=wVyZ-KveN-w>
[multicore is coming along amazingly]
<https://discuss.ocaml.org/t/the-status-of-modular-implicits/6680>
[implicits]
<https://discuss.ocaml.org/t/the-status-of-modular-implicits/6680>
[A wide variety of application domains]
<https://patricoferris.github.io/ocaml-survey-analysis/#Fluency-in-other-languages-&-Application-domains->
["other fluent languages"]
<https://patricoferris.github.io/ocaml-survey-analysis/#Fluency-in-other-languages-&-Application-domains->
[implementations]
<https://patricoferris.github.io/ocaml-survey-analysis/#OCaml-Tools->
[gen_js_api] <https://github.com/LexiFi/gen_js_api>
[bonsai] <https://github.com/janestreet/bonsai>
[Opam with the public repository]
<https://patricoferris.github.io/ocaml-survey-analysis/#OCaml-Tools->
[opam] <https://github.com/ocaml/opam/projects/1>
[community meetings]
<https://github.com/ocaml/opam/wiki/Community-dev-meetings>
[effort has been made]
<https://discuss.ocaml.org/t/what-libraries-are-missing/6543>
[communication channels]
<https://patricoferris.github.io/ocaml-survey-analysis/#Community-Interaction->
After many replies, Patrick Ferris said
───────────────────────────────────────
Thanks for the great suggestions so quickly! I'll also set aside some
time to summarise everything once some more ideas are posted and there
is general consensus.
I believe a common theme among the comments in the survey
is a lack of structure found in documentation, libraries,
and critical build tools
Completely agree – documentation is fundamental as it is felt by most
I feel. The limitations of build-tools are perhaps more felt by more
advanced or complicated workflows.
curated Opam repos
This sound really interesting – from experience I think quite a few
beginners are unfamiliar with the ability of having multiple repos. I
certainly was until I started cross-compiling to RISC-V :)) I think a
good example curated repo would help drive this home more, do you know
if one exists beyond the cross-compiling ones which aren't exactly
what we're looking for?
Use sourcegraph and other code search mechanisms to
surface all the work in progress (WIP) libraries that
people have put together on github/gitlab/etc
This sounds good. I do wonder if more
documentation/tutorials/awareness of [dune-release] and [opam-publish]
might also help. I, for one, am sitting on a few libraries I should
really publish but the process can be a little intimidating, so I
still agree a search would be great but ultimately it would be good
for libraries to make it upstream.
Thanks for this great analysis
Thanks for the kind words :))
Is GSoC still a thing?
Not in a position to start making GSoC proposals but they are
definitely still a thing! For example [two] [projects] I think are
great, still do them. Even outside of GSoC perhaps a curated list of
OCaml Community approved projects/internships that would benefit the
whole community would be good that different
businesses/organisations/foundations could use when making their own
internships would be useful? At least everyone would be united on that
front.
[dune-release] <https://github.com/ocamllabs/dune-release>
[opam-publish] <https://github.com/ocaml-opam/opam-publish>
[two]
<https://summerofcode.withgoogle.com/archive/2019/organizations/5422734538964992/>
[projects] <https://www.lowrisc.org/docs/gsoc-2020-ideas/>
gasche then said
────────────────
Note: I "split" the [excellent discussion] by @patrickoferris as a
separate topic, as it was going in the (very useful) direction of
discussing broadly the ecosystem, rather than specifically the survey
result. I would encourage people to post here for specific details on
the survey process and results, and create new topics for discussions
inspired by the survey.
Let me quote below the part of @patricoferris' post that would be most
useful to anyone interested in processing the results:
I think it's quite important to take the results from this
survey and find actionable items (or highlight ongoing
efforts) which aim to improve different problems for
different users. As a result I have put together some
graphs trying to categorise the answers based on
proficiency (`beginner', `intermediate', `advanced' and
`expert'). These groupings are quite subjective but I
thought it might offer a slightly more nuanced look into
what problems users had, where they come from, what they
are doing and what commonalities they share. The [code is
here] and the [HTML version of the notebook here] (I'm no
data-scientist nor python developer :snake: if you see
glaring mistakes do raise an issue!).
The [new topic] has excellent discussion on Patrick's findings from
the survey data.
[excellent discussion]
<https://discuss.ocaml.org/t/suggestions-from-the-ocaml-survey-result/>
[code is here]
<https://github.com/patricoferris/ocaml-survey-analysis/blob/main/survey-analysis.ipynb>
[HTML version of the notebook here]
<https://patricoferris.github.io/ocaml-survey-analysis/>
[new topic]
<https://discuss.ocaml.org/t/suggestions-from-the-ocaml-survey-result/>
Suggestions from the OCaml Survey result
════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/suggestions-from-the-ocaml-survey-result/6791/14>
Anil Madhavapeddy said in this new thread
─────────────────────────────────────────
Thanks Patrick for the fantastic distillation of results. It's
extremely useful to see our user responses segmented by their
experience. (in particular, our self-identified expert userbase runs
Coq a lot more than our self-identified newcomer codebase who tend to
use JavaScript – we want to make sure we continue to help all of these
segments!).
At a high level, this survey has already influenced the Platform tool
developers. Documentation has been identified as a priority item for
next year, and so a couple of next steps are happening this week
already among the various groups:
• The regular odoc [developer meeting] this week will also feature the
opam team as part of our [community dev meetings], and we are
getting together to put the final plan together to put a
docs.ocaml.org site together. The results of this will be on the
dev wiki as usual, so anyone interested can track progress and
suggest ideas (the video meetings are not exactly closed, but fully
open participation isn't practical given the constraints of current
technology – please ping the odoc maintainer @jonludlam directly if
interested in attending)
• the second part of a docs site is to ensure we have really reliable
and solid workflows for opam-repo contributions (including bulk
builds, health checks and having a more automated contribution
process, ideally without having to run a CLI tool). The next opam
dev meeting [later this week] will feature us planning a switch to a
[cluster-based nextgen CI] for opam-respository. We've also invited
the maintainers of the Coq opam repository as well as Tezos and Jane
Street (who contribute large package sets regularly) so we can
ensure we work well with those ecosystems as well. Our intention
here is to really reduce the burden on contributions to opam
repository by mechanising as much of the grunt work as possible,
thereby helping both beginners and expert users. Our new CI will
also feature macOS and Windows testing as we bring those cluster
workers online, and be much more easily extensible to custom
workflows.
• Having all the fancy package cluster builds in the world don't help
if noone is actually writing any documentation in their
libraries. We're hoping that new tools (such as [mdx] from Real
World OCaml) will reduce the friction of entry to writing ocamldoc
tutorials and sites. The mdx tool usage is easy but the
implementation is quite complex (due to the interlock with the
internal compiler-libs), so there is a [mdx team] working away on
it, including hopefully speeding it up with native code compilation
and continuing to improve the integration with dune and other build
tools. @yminsky and I use mdx to write the whole of Real World
OCaml (v2 of which is coming out soon in print), and we are most
eager for other people to fork our tools and write their own books
(like the [Owl scientific computing book]).
• Finally, @ashish @gemmag and I have been putting our heads together
to get funding sorted to reboot the ocaml.org site and make it
easier to maintain, in recognition of the fact that the "old guard"
(Christophe, Ashish, Phillippe, myself) just don't have the
day-to-day time anymore to keep things up. @patricoferris, @kanishka
@sanette Bella and @JohnWhitington have all been contributing
content to ocaml.org, and we are doing both incremental changes and
also overhauling the internals of how it is built to use the latest
and greatest innovations. I'm excited to see what all these new
contributors will come up with.
This is of course not a closed list of action items – simply what I am
tracking as the coordinator of the OCaml Platform efforts – so please
keep suggestions and analysis flowing.
I would suggest one good way to "wrap up" the survey is to:
• transfer @patricoferris' notebook over to the `ocaml/' GitHub org.
• once the discussions and next steps here settle (both from
individual comments, and also the aggregated decisions of the
various Platform teams like the opam, odoc and dune maintainers),
some combination of @patricoferris @xavierleroy @gasche and myself
summarise them in that notebook as priority items we would like to
accomplish next year.
• we publicise the survey results notebook and conclusions on
ocaml.org and this forum prominently.
• we figure out a way to get more people involved and contributing in
concrete projects around documentation (in particular) next year,
perhaps via schemes such as Outreachy or direct funding from
organisations such as the OCSF.
Thanks to everyone for the input and comments so far. If anyone has a
burning desire to be in any of the dev meetings, please get in touch
with me directly (anil at recoil.org) or the maintainers of the
individual tool. I've never seen this much activity happening in all
my time working on OCaml, so it warms my heart on this crisp winters
day to see all the constructive positivity and effort going on. Keep
it up and keep the suggestions coming :slight_smile:
[developer meeting] <https://github.com/ocaml/odoc/wiki>
[community dev meetings]
<https://github.com/ocaml/opam/wiki/Community-dev-meetings>
[later this week]
<https://github.com/ocaml/opam/wiki/Community-dev-meetings#20112020-opam-repo-testing-and-handling-and-bulk-builds>
[cluster-based nextgen CI]
<https://www.youtube.com/watch?v=HjcCUZ9i-ug&list=PLKO_ZowsIOu5fHjRj0ua7_QWE_L789K_f&index=2>
[mdx] <https://github.com/realworldocaml/mdx>
[mdx team] <https://github.com/realworldocaml/mdx/wiki>
[Owl scientific computing book]
<https://ocaml.xyz/book/introduction.html>
OCaml for ARM MacOS
═══════════════════
Archive: <https://discuss.ocaml.org/t/ocaml-for-arm-macos/6019/20>
Continuing this thread, John Whitington said
────────────────────────────────────────────
OCaml 4.12 from trunk builds fine on ARM macOS for me, and I can build
all my software, including mixed C/OCaml DLLs and other exotic things.
For those interested in timings, "make -j9 world.opt" for OCaml runs
in 1m6s.
TechEmpower benchmark: httpaf + lwt + unix on par with Haskell's warp
═════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/techempower-benchmark-httpaf-lwt-unix-on-par-with-haskells-warp/6788/1>
blandinw announced
──────────────────
The recent talks around this benchmark (started by @rbjorklin) and the
OCaml results piqued my curiosity. After all, one of OCaml's strengths
is its performance! It should smoke node.js and be in the same
ballpark as Haskell, Go, etc., right? Well, only one way to find out!
To start simple, I wanted to establish a baseline using a pre-forking
approach with a listening socket shared by all children and go from
there. It turns out I didn't need to go far at all. This simple
architecture was enough to leave node.js in the dust and get results
similar to Haskell and Go. This says a lot about the current quality
of OCaml and the ecosystem we have today! Handshakes all around! 🍺🐫
You can find the results for the [JSON benchmark here]. Be sure to
check the Latency tab. Note that a lot of the top performers optimize
aggressively by precomputing responses, using object pools, etc.
From my limited testing, the big difference with the previous OCaml
attempts might be that they had to use (the otherwise amazing)
`haproxy' to load-balance between all cores. Pre-forking and sharing a
socket removes that need, so that all cores can do useful work. I also
had some fun using `SIGALRM' to render the date only once per second.
As a side note, it was my first time using these UNIX APIs from OCaml
and it was an eye-opening experience to be able to leverage all that
power outside of C and without giving up any of OCaml's strengths.
I'm happy with the results, but it should be possible to improve even
further by:
• profiling with `perf' to know where time is spent, e.g. in JSON
encoding, allocations, GC, context switches, etc.
• using `libuv' instead of `libev', maybe via [this PR to Lwt]
• using [Multicore domains] to use a pre-threaded architecture as
opposed to a pre-forked architecture and hopefully reduce context
switch costs, see [Linux Applications Performance]
Contributing is pretty easy, just clone [this repo] and run `./tfb
--test httpaf --type json --concurrency-levels 512'.
[JSON benchmark here]
<https://www.techempower.com/benchmarks/#section=test&runid=032630e0-3a86-4eac-ae2d-517e8b9586ac&hw=ph&test=json&a=2>
[this PR to Lwt] <https://github.com/ocsigen/lwt/pull/811>
[Multicore domains] <https://www.youtube.com/watch?v=Z7YZR1q8wzI>
[Linux Applications Performance]
<https://unixism.net/2019/04/linux-applications-performance-part-vi-polling-servers/>
[this repo] <https://github.com/TechEmpower/FrameworkBenchmarks>
Dropbox v2 API, and JSoO bindings for GridJS and IndexedDB
══════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-dropbox-v2-api-and-jsoo-bindings-for-gridjs-and-indexeddb/6793/1>
Xavier R. Guérin announced
──────────────────────────
I am releasing on GitHub today a set of libraries under ISC:
1. `xguerin/ocaml-dropbox': Dropbox v2 API. `auth', `check', and
`files' operations are available, the rest is WIP.
2. `xguerin/ocaml-dropbox': JSoO bindings for [gridjs.io].
3. `xguerin/ocaml-indexeddb': JSoO bindings for `IndexedDB'. Based on
Thomas Leonard's work with `Irmin'.
Each of them is OPAM-enabled although not currently available in the
official repo.
[gridjs.io] <https://gridjs.io>
OCaml in Jupyter-based Environments
═══════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-in-jupyter-based-environments/6797/1>
Patrick Ferris announced
────────────────────────
As @CraigFe said, this conversation might be useful for other people
looking to run OCaml for teaching or data-science in a Jupyter-based
environments. Continuing the discussion from [Suggestions from the
OCaml Survey result] where [binder] was mentioned, this looks like a
neat way to lower the barrier to entry for programming in OCaml.
To add to what @mseri asked:
Do you know if and how one can use ocaml on colab? In
principle there is support for the jupyter protocol…
You can but it is a bit hacky. You first need to generate a blank
`ocaml.ipynb' locally and upload this to colabs, this will enable you
to select `Runtime > Change Runtime Type > Runtime Selection' to
OCaml. Then you need to install OCaml, jupyter and add the
kernel. This code block did the trick for me:
┌────
│ !add-apt-repository ppa:avsm/ppa && apt-get update && apt-get install opam m4 libgmp-dev
│ !opam init --disable-sandboxing -y
│ !opam install jupyter
│ !opam exec -- ocaml-jupyter-opam-genspec
│ !jupyter kernelspec install --user --name ocaml-jupyter "$(opam var share)/jupyter"
└────
It takes quite a while and I'm not really going to recommend it, but
it does work.
I also put together a few [dockerfiles]() with the purpose of being
used in jupyter environments for teaching. Also with an example of
using [nbgrader] although I think this still needs some work (it's
based on @kayceesrk's [blog post]). Are people using other setups for
teaching OCaml in a classroom/university setting?
As a small aside, I started doing the survey analysis in OCaml, but
found two issues:
1. CSV loading with OWL was struggling (I think it struggled with the
extra `,' in questions and also didn't like multiple headers with
the same name – other libraries I think just add the column number
to make it unique), I ended up using the [csv] library and then
making the dataframe by hand.
2. The plotting library is good, but lacks the customisability that
something like Matplotlib offers. I did start using the very good
[ocaml-matplotlib bindings] but it all became a little too much
effort… :snake:
[Suggestions from the OCaml Survey result]
<https://discuss.ocaml.org/t/suggestions-from-the-ocaml-survey-result/6791/13>
[binder] <https://mybinder.org/>
[nbgrader]
<https://github.com/patricoferris/ocaml-teaching/tree/main/teaching>
[blog post]
<https://kcsrk.info/ocaml/prolog/jupyter/notebooks/2020/01/19/OCaml-Prolog-Jupyter/>
[csv] <https://github.com/Chris00/ocaml-csv>
[ocaml-matplotlib bindings]
<https://github.com/LaurentMazare/ocaml-matplotlib/>
Anton Kochkov suggested
───────────────────────
Another project that is probably a better inspiration for OCaml-based
workflow - [Pluto.jl] that is written in Julia itself.
See [Pluto presentation (20 min) at *Juliacon 2020*].
[Pluto.jl] <https://github.com/fonsp/Pluto.jl>
[Pluto presentation (20 min) at *Juliacon 2020*]
<https://www.youtube.com/watch?v=IAF8DjrQSSk>
Anil Madhavapeddy replied to Patrick Ferris
───────────────────────────────────────────
Are people using other setups for teaching OCaml in a
classroom/university setting?
We use Jupyter and nbgrader in Cambridge for the first year
Foundations of Programming course:
<https://www.cl.cam.ac.uk/teaching/1920/FoundsCS/materials.html> (the
pdf is generated from the Jupyter export, and the content is written
in Markdown and converted to notebook format using a [modified mdx]).
It's on my todo list to resurrect
<https://github.com/andrewray/iocamljs> so that we can do this with a
pure-client-side experience. We have internal hosted servers so that
each student has their own container running their Jupyter/OCaml
kernel, but I think it would be more robust to switch entirely client
side for everything except exercise grading.
[modified mdx] <https://github.com/realworldocaml/mdx/pull/124>
Philippe asked and Anil Madhavapeddy replied
────────────────────────────────────────────
This tool shows the interpreter outputs but can also
render graphics in the page. Do you have plans to include
a similar feature in mdx? (I’m asking because if this is
the case, I’d be happy to lend a hand).
Doesn't support it, and that sounds awesome. @jonludlam did something
involving registering OCaml toplevel printers that can output HTML
that renders in a notebook (so you can pipe a `type tree = ...'
through dot to render it graphically). Never upstreamed that work I
think. Feel free to create an issue on the mdx repo – I think the
utility of it will increase dramatically if it can create notebooks!
I have to say though, writing ocaml code in a markdown
document is not very convenient since you don’t have
merlin+syntax highlighting helping you. Ideally we’d need
an environment that knows both markdown and ocaml very
well…
Yes indeed, mdx supports this mode too. In Real World OCaml, we have
all our examples as separate files that can have `dune build' run on
them, and then `mdx' supports external references as well. See for
example the [JSON chapter in RWO]: you just add a `file=' block into
the Markdown, and the dune promotion rules will either run the
toplevel or include the external ML file and update the Markdown
content automatically.
[JSON chapter in RWO]
<https://github.com/realworldocaml/book/blob/master/book/json/README.md>
Jon Ludlam said
───────────────
Registering 'rich' printers with Jupyter is a standard feature of the
wonderful [ocaml-jupyter] (thanks, @akabe!) - you just register a
printer as normal that ends up calling [Jupyter_notebook.display]. I
have a couple of examples [here] for displaying trees – the one you
mentioned – and ppm files. It would be very neat to be able to do
something similar for mdx, and probably not too tricky too. Perhaps a
good starter project?
[ocaml-jupyter] <https://github.com/akabe/ocaml-jupyter>
[Jupyter_notebook.display]
<https://akabe.github.io/ocaml-jupyter/api/jupyter/Jupyter_notebook/index.html#val-display>
[here] <https://github.com/jonludlam/focs-support/tree/master/lib>
Ocurrent/opam Docker images have moved to ocaml/opam
════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocurrent-opam-docker-images-have-moved-to-ocaml-opam/6798/1>
Thomas Leonard announced
────────────────────────
The [docker base image builder] has been reconfigured to push the
images to the `ocaml/opam' repository on Docker Hub.
If you were previously using the `ocurrent/opam' repository, you
should update to the new location. The `ocurrent/opam' repository will
not get any further updates.
The images provide many combinations of distribution, OCaml version,
architecture and flags. e.g. to get an environment with OCaml 4.11
installed on Debian 10:
┌────
│ docker run --rm -it ocaml/opam:debian-10-ocaml-4.11
└────
Or to try the 4.12 pre-release, compiled with AFL fuzzing support,
you'd use `ocaml/opam:debian-10-ocaml-4.12-afl'.
(the full set of builds can be seen at the service's page at
<https://base-images.ocamllabs.io/>)
[docker base image builder]
<https://github.com/ocurrent/docker-base-images>
Christian Lindig then said
──────────────────────────
I recently looked at `ocaml/opam:debian-10-ocaml-4.09' and noticed
that in the root of the image the Dockerfiles were included that
generated the image. I'm not sure that was always the case but found
that helpful.
ocaml-lsp-server 1.2.0
══════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ocaml-lsp-server-1-2-0/6799/1>
Rudi Grinberg announced
───────────────────────
On behalf of the ocaml-lsp team, I'd like to announce version 1.2.0.
This version contains many bug fixes and some performance improvements
A couple of interesting features made it in as well:
• Auto-completion of OCaml keywords (not available for reaso)
• The ability to jump to the declaration of a value in the .mli.
Yawar Amin added
────────────────
Release page: <https://github.com/ocaml/ocaml-lsp/releases/tag/1.2.0>
ca-certs and ca-certs-nss
═════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ca-certs-and-ca-certs-nss/6804/1>
Hannes Mehnert announced
────────────────────────
I'm pleased to announce the release two opam packages, [ca-certs] and
[ca-certs-nss], which use the trust anchors from the system / embed
trust anchors from [Netscape Security Services] (these days used by
the Mozilla Firefox browser).
For some context: when establishing a TLS connection you likely want
to verify that the server provides a valid certificate – on the open
world wide web this boils down to "is valid at the current time" and
"is signed by a trustworthy entity" (such as
<https://letsencrypt.org/> - which validates that you have access to
the domain before signing a certificate). If you do not verify the
server certificate, a person may be in the middle of the connection
and read and modify arbitrary communication content. Read more about
[this topic on Wikipedia]. NB in private setups you can use your own
CA setup and won't need ca-certs / ca-certs-nss.
Now, different operating systems store this information in different
places and formats – for Unix (and Linux) there is unfortunately no
common API or file location. To abstract over this, the package
ca-certs provides the API `val authenticator : [...] -> unit ->
(X509.Authenticator.t, [> `Msg of string ]) result' – which composes
well with OCaml-TLS API for TLS clients (that receive such an
authenticator).
The ca-certs package has initially been implemented by @emillon in
2019, and only recently been pushed to opam-repository. If you're
using a not-so-mainstream Linux distribution (or other Unix), we're
interested in your feedback: does a `dune runtest' work on your
system? – it has been tested apart from debian, ubuntu, SuSE, CentOS,
also on FreeBSD, OpenBSD, and macOS. The macOS support uses the
`security' command, and could be improved by using appropriate API
calls – there is no support for Windows at the moment (if you're
interested in contributing support for windows, [it should be pretty
straightfoward]).
The ca-certs-nss package uses the same versioning scheme as NSS, and
embeds the trust anchors that your Firefox browser has as well. This
is meant as alternative to ca-certs (e.g. if you're on a system which
is not (yet) supported by ca-certs), or in a MirageOS unikernel (where
there's no access to the host trust anchors).
We're interested in your feedback, and hope by releasing those
libraries to improve the security of network clients across the OCaml
ecosystem by providing a simple API to authenticate server
certificates. If you're running into issues, please don't hesitate to
reach out.
To install, `opam install ca-certs' / `opam install ca-certs-nss' is
all you need.
[ca-certs] <https://github.com/mirage/ca-certs>
[ca-certs-nss] <https://github.com/mirage/ca-certs-nss>
[Netscape Security Services]
<https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS>
[this topic on Wikipedia]
<https://en.wikipedia.org/wiki/Certificate_authority>
[it should be pretty straightfoward]
<https://github.com/mirage/ca-certs/issues/4>
Compatibility packages for 4.12 (either, semaphore-compat)
══════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-compatibility-packages-for-4-12-either-semaphore-compat/6806/1>
Craig Ferguson announced
────────────────────────
I'm pleased to announce the release of two small compatibility
libraries for modules added in OCaml 4.12.0:
• [`either'] – compatibility shims for the [`Either'] module, which
provides a canonical sum type in the standard library + various
utility functions for it. To use it: simply add `either' as a
library dependency and refer to the `Either' module, which will
either alias the `Either' provided by the standard library or
re-implement it as appropriate.
• [`semaphore-compat'] – compatibility shims for the [`Semaphore']
module in `systhreads', which provides binary and counting
semaphores for use in multi-threaded programs. To use it: add
`semaphore-compat' as a library dependency and `open
Semaphore_compat' in any module that requires semaphores.
*_Note on OCaml concurrency primitives_*. Users of OCaml's `Mutex'
module should beware that OCaml 4.12 features a changes to mutex
semantics on certain platforms, since the mutexes are now "error
checking" (as noted in the [changelog]). One consequence of this is
that **unlocking a mutex from a non-owning thread will now always
fail**, whereas previously this might succeed depending on the
platform's mutex implementation – notably, `glibc''s mutexes allow
this. If your code relies on this property of pre-4.12 mutexes, you
may wish to add a dependency on `semaphore-compat' and switch to using
binary semaphores instead (as these provide the right flavour of
concurrency primitive in a POSIX-compatible manner).
Opam library authors making use of these compatibility libraries in
their public API are encouraged to [conditionally depend] on them in
order to ensure that downstream users of your library don't pull in
unnecessary compatibility shims. This can be done as follows:
┌────
│ depends: [
│ ("ocaml" {>= "4.12.0"} | "either")
│ ]
└────
I hope you find these libraries useful!
[`either'] <https://github.com/mirage/either>
[`Either'] <https://github.com/ocaml/ocaml/blob/trunk/stdlib/either.mli>
[`semaphore-compat'] <https://github.com/mirage/semaphore-compat>
[`Semaphore']
<https://github.com/ocaml/ocaml/blob/trunk/otherlibs/systhreads/semaphore.mli>
[changelog] <https://github.com/ocaml/ocaml/blob/trunk/Changes#L329>
[conditionally depend]
<https://opam.ocaml.org/doc/Manual.html#Package-Formulas>
Hannes Mehnert then asked
─────────────────────────
Thanks for your work on this.
are encouraged to [conditionally depend]
While I understand at the opam level how this could work, what about
`_tags' or `dune' – there's need to repeat that condition (since in
OCaml < 4.12 you'll need `(libraries either)' in your dune file / `*:
package(either)' in your `_tags' file (+in your `META' file) – or is
there something obvious I misunderstand?
I'm asking since I'd be interested in a general strategy / howto of
such compatibility libraries. I'm also curious why there are different
approaches in respect to package name and strategy:
• stdlib-shims (seems to (attempt to) cover all stdlib changes?)
• bigarray-compat (there's a `-compat' suffix)
• seq (there's a `seq.base' for ocaml >= 4.07.0)
Are there plans to unify the approaches? I'm in favour of a
"conditional dependency if the OCaml distribution does not yet provide
the module" and "no dependency if the OCaml distribution provides that
module" – but as expressed above, I think this is needed both at the
opam and the ocamlfind / library level.
[conditionally depend]
<https://opam.ocaml.org/doc/Manual.html#Package-Formulas>
Craig Ferguson replied
──────────────────────
To begin, I'll fill out my reasons for recommending conditional
dependencies somewhat.
Compatibility libraries and unconditional dependencies
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
The issue with unconditionally depending on compatibility libraries is
that it forces users to take on that implicit dependency – even if
they don't need it – and ties that decision to the _current_ minimal
version supported by one of their dependencies. This prevents use of
`(implicit_transitive_deps false)' in those downstream libraries,
since the authors cannot know whether they actually depend on the
alias transitively. For instance:
• `lwt' currently depends on `result' unconditionally (to retain
support for `4.02.0');
• as an author of `irmin' – which goes back to `4.08.0' and so always
has access to `Stdlib.Result' – I must be able to resolve the
`Lwt_result.t' type, but I don't in general know which dependencies
are necessary to do that since it's a property of `Lwt''s code and
not mine.
• I must either add an explicit dependency on `result' – because
that's what's _currently_ necessary given `Lwt''s own dependencies –
or give up on using `(implicit_transitive_deps false)'
entirely. Either way, if I myself expose a `result' type in the API
of Irmin, this problem cascades to users of Irmin too.
For Lwt specifically, I need to find the time to upstream a patch to
fix the issue: <https://github.com/ocsigen/lwt/issues/794>.
Conditional dependencies and Dune
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
It's possible to conditionally depend on a library on the Dune side
via either a [`(select ... from ...)'] dependency or by generating
Dune files programatically (either w/ `(include dune.inc)' or
[`Jbuild_plugin']). None of these solutions are particularly
compelling to me, but I've been bitten by this problem enough to be
willing to consider ugly solutions – at least for very widely-used
libraries such as Alcotest.
It's possible that conditional dependencies are the wrong approach
altogether and we should be using something like [`(re_exports)']
instead. I'm not familiar with that feature, but it looks like it
might work to avoid the above issues. Perhaps a Dune maintainer could
weigh in here, and I'll happily remove my suggestion re. conditional
dependencies for something better :slightly_smiling_face:
[`(select ... from ...)']
<https://dune.readthedocs.io/en/stable/concepts.html#alternative-dependencies>
[`Jbuild_plugin']
<https://dune.readthedocs.io/en/stable/advanced-topics.html#ocaml-syntax>
[`(re_exports)'] <https://github.com/ocaml/dune/pull/2605>
Different approaches to compatibility libraries
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
I’m asking since I’d be interested in a general strategy /
howto of such compatibility libraries. I’m also curious
why there are different approaches in respect to package
name and strategy:
• stdlib-shims (seems to (attempt to) cover all stdlib
changes?)
• bigarray-compat (there’s a `-compat' suffix)
• seq (there’s a `seq.base' for ocaml >= 4.07.0)
Indeed, there are quite a few different mechanisms used for this. Even
just for this release, `either' and `semaphore-compat' use different
techniques! Partly this is because it's easier to shim modules in
`Stdlib' since it is the default-opened library. In particular,
`Either' can exist without a namespace collision, and the
_implementation_ of `either' can refer to `Stdlib.Either' to avoid
Dune getting confused about a self-referential module. This is the
approach that was taken with the `result' compatability package too.
Unfortunately, the above factors don't apply to `Semaphore', which is
why I grudgingly settled on `semaphore-compat' instead. I'm not sure
why the `Bigarray' compat module didn't take the more "direct"
approach, but I was happy to be consistent with it.
One final point: the `stdlib-shims' library exists _solely_ to provide
the alias `Stdlib' ↦ `Pervasives' in order to compensate for the name
change. It doesn't ever provide re-implementations of the modules and
functions that were added since (and so really is a `-shim' and not a
`-compat'). A user of `stdlib-shims' is still restricted to the set of
stdlib functions provided by their minimal supported OCaml version.
Jupyter with pyml
═════════════════
Archive: <https://discuss.ocaml.org/t/jupyter-with-pyml/6813/1>
n4323 asked
───────────
Hi, I would like to achieve something similar to, but different from
what is discussed in
<https://discuss.ocaml.org/t/ocaml-in-jupyter-based-environments/6797>
. tl; dr: How can I get python REPL access to OCaml values exported
via `pyml'?
The use case is this: I have an OCaml library that does
machine-learning type computation. I would like to generate, plot and
analyze the results interactively, with the best possible ease of use
and flexibility. For the plotting and analysis I want to use Python.
I have tried `ocaml-jupyter', which provides a nice notebook
environment but no python kernel and the very useful
`ocaml-matplotlib' which offers a matplotlib binding but cannot cover
all my python library needs.
A possible solution would be to use `pyml' and its numpy array
memmapping support to export OCaml values to a python process. The
problem here is that I can't find a way to interact with the python
process spawned from `pyml' in an interactive python console. I am
thinking there should be some way to spawn a Jupyter python kernel
from `pyml' and then access that from a separate jupyter console. I
have not been able to find one by googling though.
Another solution might be to wrap my OCaml library as an external
python module. I'm afraid that this may be painful – please convince
me otherwise ;)
Craig Ferguson
──────────────
You may be interested in this post on the Jane Street tech blog:
[*Using Python and OCaml in the same Jupyter notebook*],
by Laurent Mazare (@laurent).
which offers some general tips and some bespoke tooling (showcased in
<https://colab.research.google.com/drive/1MDuZc0v60lzBg0_xeiPR-bB6dCIWxvzT>). I
have found this very useful in the past when trying to do Python +
OCaml interop in Jupyter notebooks.
[*Using Python and OCaml in the same Jupyter notebook*]
<https://blog.janestreet.com/using-python-and-ocaml-in-the-same-jupyter-notebook/>
OCaml needs an arbitrary-precision decimal type
═══════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-needs-an-arbitrary-precision-decimal-type/6735/6>
Yaron Minsky announced
──────────────────────
Our Bigdecimal library is now live:
<https://github.com/janestreet/bigdecimal>
It won't be in the ordinary Opam repo until our next stable release,
which is scheduled for early 2021.
Type-at-point ocaml-lsp/merlin in vim/neovim
════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/type-at-point-ocaml-lsp-merlin-in-vim-neovim/6832/1>
Nicolas Gimenez discussed
─────────────────────────
I'd like to know whether ocaml-lsp supports type-at-point feature? The
point is to display to the user the type at point inferred by the
compiler. I think this functionality already exists in Merlin for a
while: [merlin-show-type-at-point], and I've read [articles] talking
about people's corresponding lsp-mode/merlin Emacs config. How about
VIM? Is it integrated in ocaml-lsp already? From looking on GitHub, it
seems it's not, but then how come people manage to integrate it in
Emacs' lsp-mode?
[merlin-show-type-at-point] <https://github.com/ocaml/merlin/issues/6>
[articles] <https://khady.info/emacs-ocaml-lsp.html>
mudrz replied
─────────────
I am using `neovim' and the following config:
in `.vimrc'
┌────
│ Plug 'sheerun/vim-polyglot'
│ Plug 'neoclide/coc.nvim', {'branch': 'release'}
│
│ nnoremap <silent> gh :call <SID>show_documentation()<CR>
│
│ function! s:show_documentation()
│ if (index(['vim','help'], &filetype) >= 0)
│ execute 'h '.expand('<cword>')
│ else
│ call CocAction('doHover')
│ endif
│ endfunction
└────
and in `:CocConfig':
┌────
│ "languageserver":{
│ "ocaml": {
│ "command": "ocamllsp",
│ "rootPatterns": ["dune-project"],
│ "filetypes": ["ocaml"],
│ "initializationOptions": {},
│ "settings": {}
│ }
│ }
└────
which allows to show a type tooltip like this when you press `gh`
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/9/943c5e3dbeb3f4d92bfc905a57d9ef2cfc339317_2_1380x150.png>
Raphaël Proust also replied
───────────────────────────
In my `.config/nvim/init.vim' I have the default things setup by opam
and then (amongst other things):
┌────
│ autocmd FileType ocaml nnoremap <LocalLeader>t :MerlinTypeOf<CR>
└────
ocamlearlybird now an OCaml Software Foundation supported project
═════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-ocamlearlybird-now-an-ocaml-software-foundation-supported-project/6834/1>
文宇祥 announced
────────────────
I’m happy to announce ocamlearlybird now an OCaml Software Foundation
supported project.
We will continue to develop/maintain ocamlearlybird, and may
contribute to ocamlrun/ocaml to improve/achieve OCaml bytecode/native
debugging support.
Please expect that OCaml debugging experience will be as good as
Javascript in the future.
Louis Roché then said
─────────────────────
I suppose that you are referring to
<https://github.com/hackwaly/ocamlearlybird>
That's a great news. Thanks for your work.
release of mc2 0.1, a SMT solver
════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-release-of-mc2-0-1-a-smt-solver/6835/1>
Simon Cruanes announced
───────────────────────
I'm happy to announce the initial release 0.1 of [mc2], a [SMT solver]
in pure OCaml based on the mcSAT framework. It can handle the QF_UFLRA
fragment (functions and linear rational arithmetic). It is more of a
research project than a production ready solver, but has pretty
acceptable performance (see for examples benchmarks on [QF_UFLRA] and
[QF_LRA] with timeout=10s).
The solver is Apache licensed and weights around 7kloc (+ a few basic
dependencies). Questions or comments welcome :slight_smile:
[mc2] <https://github.com/c-cube/mc2>
[SMT solver]
<https://en.wikipedia.org/wiki/Satisfiability_modulo_theories>
[QF_UFLRA]
<https://benchpress.cedeela.fr/show/res-20201117T131950-cfa5920b-5187-4d73-8f5b-00b5c77aaeb8.sqlite>
[QF_LRA]
<https://benchpress.cedeela.fr/show/res-20201116T203511-687d557c-5c27-4478-9cc2-8b2603820ab6.sqlite>
Wednesday, 25th November 2020 is MirageOS Bug Cleaning Day!
═══════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-wednesday-25th-november-2020-is-mirageos-bug-cleaning-day/6836/1>
Hannes Mehnert announced
────────────────────────
We have many repositories that have lots of old and no-longer-relevant
issues in them (some have even been fixed!) as well as issues that
haven't gotten a reply yet from a maintainer. In preparation for the
next release, I think it would be nice to take a day and do some
housecleaning.
Our last bug cleaning day was on 17th November 2017, and pretty
successful.
On Wednesday, 25th November (in two days) most people of the
mirage-core team will be going through old issues and coordinating our
efforts on the #mirage channel over on irc.freenode.net. I expect
there to be the most activity during 10:00-18:00 UTC and maybe a bit
later, but don't feel limited to that timeslot – if you're familiar
with a repository and have a bit of time, we'd love your help any time
at all. Please do join us if you're free! We'll as well be watching
the OCaml discord #mirage channel
<https://discord.com/channels/436568060288172042/519199441769594911>
If you're not sure where to start, here's a link to a GitHub search
for all issues in repositories owned by the Mirage organization which
are open and not archived, sorted with the least recently updated
first, for your editing and browsing pleasure:
<https://github.com/issues?q=is%3Aopen+is%3Aissue+org%3Amirage+archived%3Afalse+sort%3Aupdated-asc>
If you've a specific issue at your heart that you want to be solved,
please show up and tell us about it.
Other OCaml News
════════════════
>From the ocamlcore planet blog
──────────────────────────────
Here are links from many OCaml blogs aggregated at [OCaml Planet].
• [Editor Support, Custom Operators and More]
• [Coq 8.12.1 is out]
[OCaml Planet] <http://ocaml.org/community/planet/>
[Editor Support, Custom Operators and More]
<https://rescript-lang.org/blog/editor-support-custom-operators-and-more>
[Coq 8.12.1 is out] <https://coq.inria.fr/news/coq-8-12-1-is-out.html>
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <http://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <http://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <http://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20201124/f857077d/attachment-0001.html>
More information about the caml-news-weekly
mailing list