[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Aug 3 02:11:35 PDT 2021
Hello
Here is the latest OCaml Weekly News, for the week of July 27 to August
03, 2021.
Table of Contents
─────────────────
OCaml Café: Wed, Aug 4 @ 7pm (U.S. Central)
Do You Need `ppxlib' and `ocaml-migrate-parsetree' in the Same Project?
first release of FASMIFRA (a molecular generator)
Become an Outreachy Mentor: support the growth and diversity of the OCaml community
Alt-Ergo 2.4.1
Set up OCaml v1.1.12
Set up OCaml 2.0.0-beta3
JSOO build integration with JS front-end project?
Multicore OCaml: July 2021
Old CWN
OCaml Café: Wed, Aug 4 @ 7pm (U.S. Central)
═══════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ocaml-cafe-wed-aug-4-7pm-u-s-central/8169/2>
Michael Bacarella announced
───────────────────────────
Reminder: this is in 2 days!
Do You Need `ppxlib' and `ocaml-migrate-parsetree' in the Same Project?
═══════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/do-you-need-ppxlib-and-ocaml-migrate-parsetree-in-the-same-project/8207/1>
Sonja Heinze announced
──────────────────────
Hello PPX users (directly or transitively),
I'd like to discuss here in this thread if there's still a demand to
use `ppxlib' and `ocaml-migrate-parsetree.1.x' (`OMP1') in the same
project. Yesterday, I posted here on discuss about the state of the
PPX ecosystem and [pointed out that the public PPX ecosystem has
almost completely shifted to `ppxlib']. Still, we'd like to know your
perspective on this question. Have you lately run into a problem due
to the incompatibility between `ppxlib' and `OMP1'? Did you formerly
have such problems but lately all the PPXs you use have already been
ported to `ppxlib'? These questions are directed to any OCaml user, so
please comment any details on this post and point out which PPXs were
involved. Thanks in advance!
The rest of this post gives some more context about why we're asking
these questions now in case folks are interested, but it's not needed
for the discussion we'd like to have about the questions stated above.
[pointed out that the public PPX ecosystem has almost completely shifted
to `ppxlib']
<https://discuss.ocaml.org/t/an-update-on-the-state-of-the-ppx-ecosystem-and-ppxlib-s-transition/8200#part-three-porting-ppxs-to-put-an-end-to-the-split-world-situation-5>
Some Context
╌╌╌╌╌╌╌╌╌╌╌╌
The next `ppxlib' release `0.23.0' will contain the integration of
`Astlib' and therefore will drop its dependency on
`ocaml-migrate-parsetree' (`OMP'). Now we need to decide if to mark
`OMP1' as a conflict for `ppxlib' in opam or not.
To give more background, the PPX ecosystem is [transitioning to be
more up-to-date]. The three contextually relevant states of this
transition are:
1. `ppxlib' depends on `OMP < 2.0.0'. The PPX driver is in `OMP'.
2. `ppxlib' depends on `OMP >= 2.0.0'. The PPX driver is in `ppxlib'.
3. `ppxlib' doesn't depend on `OMP' anymore. It uses its internal /
the compiler library `Astlib'. The PPX driver is in `ppxlib'.
During State 1, you can use `ppxlib' PPXs and `OMP1' PPXs in the same
project. During State 2, `ppxlib' opam conflicts with `OMP1'. During
State 3, if we leave everything as-is, you can install both `ppxlib'
and `OMP1' for the same project. But also during State 3, you're
unable to use `ppxlib' PPXs and `OMP1' PPXs in the same library or
executable because if your build system uses a PPX driver, the PPX
driver needs to be unique. Now we need to decide whether to leave the
conflict between `ppxlib' of State 3 and `OMP1' at the level of the
build system or to lift it up to the package manager by making `OMP1'
an opam conflict for `ppxlib'. Both options have their pros and cons,
which have already been somewhat discussed on [the corresponding
PR]. To sum them up:
[transitioning to be more up-to-date]
<https://discuss.ocaml.org/t/an-update-on-the-state-of-the-ppx-ecosystem-and-ppxlib-s-transition/8200/1>
[the corresponding PR] <https://github.com/ocaml-ppx/ppxlib/pull/255>
Option 1: not making `OMP1' a conflict for `ppxlib'
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
*Advantage*: You would be able to use `ppxlib' PPXs and `OMP1' PPXs in
the same project, but they still can't be used together inside the
same library or executable.
*Disadvantage*: If you use several PPXs inside the same library or
executable, there's no direct guarantee that you and your users
install compatible versions of the PPXs for the build system. To
guarantee that, you'd have to add certain dependency constraints to
your opam file. We would make sure the way to define those dependency
constraints would be well documented and could be found easily.
Option 2: making `OMP1' a conflict for `ppxlib'
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
*Advantage*: Without adding any dependency constraints yourself, opam
will ensure that you and your users don't install both `ppxlib' and
`OMP1', only to get an error at build time.
*Disadvantage*: You couldn't use `ppxlib' PPXs and `OMP1' PPXs in the
same project, not even separating them at the level of
libraries/executables. You also couldn't use an exectubale based on
`ppxlib' together with a `OMP1' PPX in the same project. In short: the
situation would stay the same as it is currently during State 2.
To make the best decision, we need to know if the possibility to use
`ppxlib' and `OMP1' inside the same project, separated by
libraries/executables, is still in demand.
Lucas Pluvinage replied
───────────────────────
In a web project I wanted to use `graphql-ppx'
(<https://github.com/reasonml-community/graphql-ppx>) but the last
version released on opam (v1.0.1) was using `OMP' whereas other parts
of the project were using ppxlib.
However the compatibility is not a hard requirement for me because
more recent versions of that package (v1.2.2) are actually compatible
with ppxlib. The solution for me was to vendor the project and wait
for an upcoming opam release, which has been discussed in a Github
issue.
Sonja Heinze then said
──────────────────────
By the way, the `graphql_ppx' version containing the `ppxlib' port is
now also [released on opam], so hopefully no one will have to vendor
it again for PPX incompatibility reasons. :)
[released on opam] <https://opam.ocaml.org/packages/graphql_ppx/>
first release of FASMIFRA (a molecular generator)
═════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-first-release-of-fasmifra-a-molecular-generator/8212/1>
UnixJunkie announced
────────────────────
If you are not a computational chemist / chemoinformatician, you will
not be interested by the following.
If you are and you are interested by machine-learning for molecular
generation, then be welcome! :slight_smile:
FASMIFRA: Fast Molecular Generation by Assembly of (Deep)SMILES
Fragments <https://github.com/UnixJunkie/FASMIFRA>
Become an Outreachy Mentor: support the growth and diversity of the OCaml community
═══════════════════════════════════════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/become-an-outreachy-mentor-support-the-growth-and-diversity-of-the-ocaml-community/8213/1>
Patrick Ferris announced
────────────────────────
What is Outreachy?
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
[Outreachy] is an initiative that provides financial support for
remote internships, in open-source communities, to under-represented
groups. It provides interns with the opportunity for their first
experience in open-source development.
Outreachy promotes diversity and inclusion in the open-source
community. It acts as a central point for organising internships for
groups of people who historically have faced systemic bias in the
programming world.
[Outreachy] <https://www.outreachy.org/>
Outreachy and OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Following the successful effort from the OCaml Software Foundation
last year, this year OCaml Labs and Tarides joined forces to continue
our efforts to bring more new developers into the OCaml community from
diverse backgrounds. We selected three Outreachy interns to work with
us on different OCaml projects.
Current OCaml Interns
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
We merged over 75 PRs and resolved over 70 issues during the initial
contribution period this year on the [ocaml/ocaml.org] repository!
This was the most activity the repository had seen since 2014. There
were so many candidate interns, keen to join the OCaml community and
start developing.
This year, we have three interns working with us: Diksha Gupta,
Odinaka Joy and Shreya Kumari Gupta.
• *Diksha Gupta* is working on automating and expanding ocaml.org's
new and experimental peertube instance for hosting OCaml-related
video content.
• *Odinaka Joy* is prototyping and building an online package search
web application and service.
• *Shreya Kumari Gupta* is helping to build and design a new ocaml.org
website.
They've all made amazing progress and we can't wait to share more
about these projects towards the end of their internships!
[ocaml/ocaml.org] <https://github.com/ocaml/ocaml.org>
Continuing to grow Outreachy and OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Outreachy programs run every six months, and so even though the
current round is still ongoing, we need to plan how to participate in
the forthcoming Winter 2021 round.
One of the big bottlenecks in growing the program is having a
sufficient supply of projects and mentors from the OCaml community who
are willing to work with the enthusiastic Outreachy interns to get
them started on contributing to our ecosystem. That's the goal of this
post – to convince you to sign up!
Benefits of being a mentor
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Outreachy is an amazing and rewarding experience that:
• Increases diversity in the OCaml community.
• Attracts more contributers to OCaml or to open-source projects in
general.
• Improves your project with an additional developer.
• Has an enriching experience: work together with people from many
different backgrounds and ways of thinking.
• Promotes OCaml: When people learn about Outreachy, they often go
through the list of communities that are or were signed up in the
past. That's a good way for those from under-represented groups to
learn about OCaml. It also puts the OCaml community in a good
light. We know that this is how some people who now form part of the
OCaml community first learned about OCaml and got excited about it.
Expectations of being a mentor
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
* Time commitment
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
During the three month internship, mentors need to commit to working
with the Outreachy intern for approximately 5 hours per week. During
the one month contribution period that takes place before the
internship starts, mentors must help prospective applicants with
issues and PRs which can take anywhere between 1 to 15 hours per week
depending on how many interested applicants there are.
* Your open-source repository
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
Most Outreachy projects that interns work on have a single, main
repository that code will eventually be merged into. It can be any
OCaml, open-source repository of code with some form of issue
tracker. For example, this summer we started with the [ocaml.org
repository]. The best repositories tend to be those that are
well-maintained, make good use of the issue tracker, and have clear
documentation that describes how to get started.
Note that although companies can take part, the open-source project
that is part of Outreachy must be in the public interest. This
generally means that OCaml libraries and tools of wider interest are
fine targets for Outreachy contributions, but not OCaml codebases that
are commercial or very specific in nature.
[ocaml.org repository] <https://github.com/ocaml/ocaml.org>
* Outreachy projects
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
[Outreachy defines a good project] for an intern as being
well-defined, self-contained, uncontroversial and incremental. When
thinking of a suitable project it is important to bear these
conditions in mind.
Mentors can support the success of the intern by preparing, in
advance, a three-month outline of the timeline the intern will
follow. This timeline divides the project into manageable chunks that
can be completed each week.
[Outreachy defines a good project]
<https://www.outreachy.org/docs/community/#project-guidelines>
* Interacting with Outreachy interns
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
The interns' backgrounds may vary. Some might be experienced
developers but most won't already know OCaml and may have questions
concerning the OCaml set-up or the langauge in general. However, the
project doesn't need to be super easy: many interns who started
without 100% skill fit have been very successful.
Also, for most interns it's the first time contributing to open-source
and they've all faced systemic bias in the past. So it's important to
provide a friendly working atmosphere. This means being patient and
making them feel comfortable to ask questions.
Step-by-step process for being a mentor
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
The Outreachy process from the mentor's perspective, including dates
for the coming round is:
<https://i.imgur.com/cI5GvF7.png>
1. The Outreachy process starts when the call for *mentoring
communities sign-up* opens. OCaml is already included in the next
Outreachy round.
2. The next step is to [*submit Outreachy projects*] to the OCaml
community on the Outreachy site. That's the first step that you, as
a potential mentor, would need to undertake. We've explained in the
section above what a project can look like. At this stage, you also
need to mention the skills that interns require to contribute to
your project. This year's list of projects with their respective,
required skills are [available online]. If you set your
requirements too low, you might be overwhelmed by the number of
applicants to your project; if you set it too high, you might not
get any applicants. The section above, _Interacting with Outreachy
interns_ describes the experience level you can expect.
3. If your project is approved, you should *prepare it for the
_contribution period_*. At this stage, it's a good to review your
contribution documentation and to label some of your project's
issues as `good-first-issue` resp. `medium`.
4. The one month *_contribution period_* is the time during which
Outreachy applicants, who have passed an _initial application_
process that's validated by the Outreachy organizers, can apply for
a defined project by contributing to its repository in the same way
that any other first-time contributer would contribute
(independently from the internship project). This involves choosing
issues, working on them and opening PRs, commenting on issues or
opening new ones, and so on. During this stage, you need to start
providing guidance and help. In particular, having
`good-first-issue`'s is imperative since it allows the participants
to become familiar with the code base. You must also have some
`medium` sized issues or tasks for participants because it allows
you to assess their application with more substantial
contributions.
5. When the contribution period ends, you need to *select (an)
intern(s)* out of the pool of applicants. To help you get an
overview, each of them will submit an application consisting of the
following two parts: a summary of their contributions to your
repository during the contribution period and the [final
application] which includes a timeline for the internship
project. The applicants are supposed to work out the timeline
together with their mentor(s), so they might ask you for guidance
and feedback in advance.
6. The interns will then be officially announced by Outreachy, and
they start working with you a few weeks later, marking the start of
their 13-week *internship period*!
[*submit Outreachy projects*]
<https://www.outreachy.org/communities/cfp/ocaml/>
[available online]
<https://www.outreachy.org/outreachy-may-2021-internship-round/>
[final application]
<https://www.outreachy.org/docs/applicant/#final-application>
Become a mentor
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Mentoring an Outreachy intern is a very enriching experience! So if
you have ideas, [submit your project on the Outreachy website], under
the OCaml community, and apply to be a mentor or co-mentor.
We are looking forward to your participation in the next Outreachy
round(s) and would love to hear your ideas (comment on the post)!
This post was written by @gs0510, @patricoferris and @pitag with lots
of useful input from @avsm and @susanstocks.
[submit your project on the Outreachy website]
<https://www.outreachy.org/communities/cfp/ocaml/>
Alt-Ergo 2.4.1
══════════════
Archive: <https://discuss.ocaml.org/t/ann-alt-ergo-2-4-1/8217/1>
OCamlPro announced
──────────────────
We are pleased to announce a new release of Alt-Ergo! Alt-Ergo 2.4.1
is now available on our [Github].
This release contains some major novelties:
• Improvement of term purification
• Implementation of a semantic term construction cache
• Replacement of Travis-CI by GitHub actions
• Improvement of documentation
• Unsoundness fixes
Do not hesitate to report bugs, to ask questions, or to give your
feedback!
[Github] <https://github.com/OCamlPro/alt-ergo/releases/tag/2.4.1>
Set up OCaml v1.1.12
════════════════════
Archive: <https://discuss.ocaml.org/t/ann-set-up-ocaml-v1-1-12/8223/1>
Sora Morimoto announced
───────────────────────
Fixed
╌╌╌╌╌
• Stop installing `depext' package on macOS runners (opam is now at
2.1.0).
<https://github.com/ocaml/setup-ocaml/releases/tag/v1.1.12>
Set up OCaml 2.0.0-beta3
════════════════════════
Archive:
<https://discuss.ocaml.org/t/ann-set-up-ocaml-2-0-0-beta3/8224/1>
Sora Morimoto announced
───────────────────────
Changed
╌╌╌╌╌╌╌
• Use the week number to manage Cygwin cache.
Fixed
╌╌╌╌╌
• Set repository priorities correctly for multiple repositories
feature.
• Lock the version of opam to be installed only to < 2.1 releases
until opam 2.2 is released.
<https://github.com/ocaml/setup-ocaml/releases/tag/v2.0.0-beta3>
JSOO build integration with JS front-end project?
═════════════════════════════════════════════════
Archive:
<https://discuss.ocaml.org/t/jsoo-build-integration-with-js-front-end-project/8226/1>
currycomb asked
───────────────
I want to use JSOO to build a JS package that contains core logic that
will run in a browser, presenting a JSON-y API to be consumed by JS
front-end code. This seems like a good way to use js_of_ocaml: all
the complicated JS UI stuff doesn't need to be replicated in OCaml,
but you can still keep your core domain logic there.
I've got dune building my .js using JSOO and exposing functions to JS.
If I copy that JS by hand to my front-end project, it works. Now I
want to be able to:
• Expose the output as something that my front-end project can bundle
without a manual copying step
• Be able to make changes to in a dev environment and see the changes
quickly in a browser
To do this with ReScript compiling my core `.ml' syntax project, I
have `esbuild' running under a watcher + `rescript build -w'. I run
`rescript build -w' on my front-end, with the core project ~npm
link~ed in. Any change I make to either project (core or front-end)
shows up effectively immediately in the browser.
Has anybody got advice on how to get a similar setup working with JSOO
compiling the core project in place of ReScript, and JS on the front
end? Ideally on github, but otherwise hints appreciated!
Patrick Ferris replied
──────────────────────
I took a stab at implementing something that could maybe be helpful,
apologies if I misinterpreted what you were asking, the repository is
[jsoo-reload-example]. I think the README explains it quite well, but
for completeness I'll add the general idea below.
Expose the output as something that my front-end project
can bundle without a manual copying step
To achieve this I used the a dune rule with a copy action. The
dependency is the output of the jsoo build (`main.bc.js') and the
target is a new js file (`print.js') in my project directory. It is
promoted automatically into that file and so if the jsoo is
recompiled, this copy happens right afterwards. I didn't commit the
file but you could.
Be able to make changes to in a dev environment and see
the changes quickly in a browser
Now thanks to the previous setup running `dune build -w' will
recompile the jsoo and copy it over whenver you make a change to files
your jsoo project requires.
All of the next part may not apply to your use case, but others might
find it somewhat useful. I used the [parcel] bundler (this is the only
bundler I know vaguely how to use) to then take the project and bundle
it into a `dist' directory. Running `parcel watch site/index.html'
concurrently with `dune build -w' now sets up a whole pipeline from
making a change to the jsoo to having a fresh, bundled output.
The final step for me (and again this probably doesn't apply) was to
have live-reloading server serve the contents of the bundled
output. Using [dream], [dream-livereload] and [irmin-watcher] (an
OCaml library for watching the filesystem) I was able to achieve this
(I'll leave out the details but everything is in the `server'
directory), so now we have:
1. `dune build -w' notices a change to the jsoo and recompiles.
2. `dune' also then copies the jsoo output over to the project
directory (in the example called `site').
3. The `site' directory has now changed triggering `parcel' to bundle
the output to `dist'.
4. `dist' has now changed triggering `irmin-watcher' which using
`dream' and `dream-livereload' uses websockets to trigger a browser
refresh.
That list should also set your expectations for how fast this all
should be… not blazingly fast, but sure beats manually copying and
restarting things by hand, hope it helps :))
[jsoo-reload-example]
<https://github.com/patricoferris/jsoo-reload-example>
[parcel] <https://parceljs.org/>
[dream] <https://github.com/aantron/dream>
[dream-livereload] <https://github.com/tmattio/dream-livereload>
[irmin-watcher] <https://github.com/mirage/irmin-watcher>
Multicore OCaml: July 2021
══════════════════════════
Archive:
<https://discuss.ocaml.org/t/multicore-ocaml-july-2021/8232/1>
Anil Madhavapeddy announced
───────────────────────────
Welcome to the July 2021 [Multicore OCaml] monthly report! This
month's update along with the [previous updates] has been compiled by
me, @ctk21, @kayceesrk and @shakthimaan. As August is usually a period
of downtime in Europe, the next update may be merged with the
September one in a couple of months (but given our geographically
diverse nature now, if enough progress happens in August I'll do an
update).
The overall status of the multicore efforts are right on track: our
contributions to the next OCaml release have been [incorporated in
4.13.0~alpha2], and our focus remains on crushing incompatibilities
and bugs to generate domains-only parallelism patches suitable for
upstream review and release. As a lower priority activity, we
continue to develop the experimental "effects-based" IO stack, which
will feature in the upcoming virtual OCaml Workshop at ICFP in August
2021.
The `4.12.0+domains' trees continue to see a tail of bugs being
steadily fixed. After last month's call, we saw a number of external
contributors step up to submit fixes in addition to the multicore and
core OCaml teams. We would like to acknowledge and thank them!
• @emillon (Etienne Millon) for running the Jane Street `core' v0.14
test suite with 4.12.0+domains and sharing the test results (and
finding a [multicore GC edge case bug] while at it).
• @Termina1 (Vyacheslav Shebanov) for testing the compilation of
`batteries' 3.30 with Multicore OCaml 4.12.0+domains.
• @nbecker (Nils Becker) for reporting on `parallel_map' and
`parallel_scan' for domainslib.
• Filip Koprivec for identifying a memory leak when using `flush_all'
with `ocamlc' with 4.12.0+domains.
All of these fixes, combined with some big-ticket compatibility
changes (listed below) are getting me pretty close to using
4.12.0+domains as my daily OCaml opam switch of choice. I encourage
you to also give it a try and report (good or bad) results on [the
multicore OCaml tracker]. If these sorts of problems grab your
attention, then [Segfault Systems is hiring in India] to work with
@kayceesrk and the team there on multicore OCaml.
For benchmarking, the Jupyter notebooks for the Sandmark nightly
benchmark runs have been updated, and we continue to test the Sandmark
builds for the 4.12+ variants and 4.14.0+trunk. Progress has been made
to integrate `current-bench' OCurrent pipeline with the Sandmark 2.0
-alpha branch changes to reproduce the current Sandmark functionality,
which will allow GitHub PRs to be benchmarked systematically before
being merged.
As always, the Multicore OCaml ongoing and completed tasks are listed
first, which are then followed by the updates from the Ecosystem
libraries. The Sandmark nightly build efforts, benchmarking updates
and relevant current-bench tasks are then mentioned. Finally, the
update on the upstream OCaml Safepoints PR is provided for your
reference.
[Multicore OCaml] <https://github.com/ocaml-multicore/ocaml-multicore>
[previous updates] <https://discuss.ocaml.org/tag/multicore-monthly>
[incorporated in 4.13.0~alpha2]
<https://discuss.ocaml.org/t/ocaml-4-13-0-second-alpha-release/8164>
[multicore GC edge case bug]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/624>
[the multicore OCaml tracker]
<https://github.com/ocaml-multicore/ocaml-multicore/issues>
[Segfault Systems is hiring in India]
<https://segfault.systems/careers.html>
Multicore OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
CI Compatibility
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#602] Inclusion of most of OCaml
headers results in requiring pthread
The inclusion of multiple nested header files requires `pthread' and
the `decompress' testsuite fails.
• [ocaml-multicore/ocaml-multicore#607] `caml_young_end' is not a
`value *' anymore
An inconsistency observed in the CI where `caml_young_end' is now a
`char *' instead of `value *'.
[ocaml-multicore/ocaml-multicore#602]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/602>
[ocaml-multicore/ocaml-multicore#607]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/607>
Crashes
┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#608] Parmap testsuite crash
`Parmap' is causing a segfault when its testsuite is run against
Multicore OCaml.
• [ocaml-multicore/ocaml-multicore#611] Crash running Multicore binary
under AFL
The `bun' package crashes with Multicore OCaml 4.12+domains, but,
builds fine on 4.12.
[ocaml-multicore/ocaml-multicore#608]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/608>
[ocaml-multicore/ocaml-multicore#611]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/611>
Package Builds
┈┈┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#609] lablgtk's example segfaults
An ongoing effort to compile lablgtk with OCaml and Multicore OCaml.
• [ocaml-multicore/ocaml-multicore#624] core v0.14: test triggers a
segfault in the GC
A segfault caused by running `core.v0.14' test suite with Multicore
OCaml 4.12.0+domains as reported by @emillon.
• [ocaml-multicore/ocaml-multicore#625] Cannot compile batteries on
OCaml Multicore 4.12.0+domains
An effort by Vyacheslav Shebanov (@Termina1) to compile
`batteries.3.30' with Multicore OCaml 4.12.0+domains variant.
[ocaml-multicore/ocaml-multicore#609]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/609>
[ocaml-multicore/ocaml-multicore#624]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/624>
[ocaml-multicore/ocaml-multicore#625]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/625>
Upstream
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#573] Backport trunk safepoints PR
to multicore
The Safepoints implementation is being backported to Multicore
OCaml. The initial test results of running Sandmark on a large Xen2
box are shown below:
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/8/814e446af614fd2bb502af0c3b79be239024a97d_2_916x1000.png>
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/5/5247895927d8e79619e1d6eb4682c2f8218d4ede_2_916x1000.png>
• [ocaml-multicore/ocaml-multicore#617] Some of the compatibility
macros are not placed in the same headers as in upstream OCaml
The introduction of a compatibility layer for GC statistics need to
be consistent with trunk.
• [ocaml-multicore/ocaml-multicore#618] Review io.c for thread-safety
and add parallel tests
The thread-safety fixes in io.c requires a review and additional
tests need to be added for the same.
• [ocaml-multicore/ocaml-multicore#623] Exposing
`caml_channel_mutex_*' hooks
A draft PR to support `caml_channel_mutex_*' interfaces from trunk
to Multicore OCaml.
[ocaml-multicore/ocaml-multicore#573]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/573>
[ocaml-multicore/ocaml-multicore#617]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/617>
[ocaml-multicore/ocaml-multicore#618]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/618>
[ocaml-multicore/ocaml-multicore#623]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/623>
Sundries
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#610] Add std gnu11 common cflags
The configure.ac file has been updated to use `-std=gnu11' in
`common_cflags' for both GCC and Clang.
• [ocaml-multicore/ocaml-multicore#614] Destroy channel mutexes after
fork
A discussion on resetting and reinitializing mutexes after fork in
the runtime.
• [ocaml-multicore/ocaml-multicore#616] Expose functions to program
with effects
A draft PR to enable programmers to write programs that use effects
without explicitly using the effect syntax.
• [ocaml-multicore/ocaml-multicore#619] Set resource Limit
A query to use `setrlimit' in Multicore OCaml, similiar, to
`Core.Unix.RLimit.set' from Jane Street's core library.
[ocaml-multicore/ocaml-multicore#610]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/610>
[ocaml-multicore/ocaml-multicore#614]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/614>
[ocaml-multicore/ocaml-multicore#616]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/616>
[ocaml-multicore/ocaml-multicore#619]
<https://github.com/ocaml-multicore/ocaml-multicore/issues/619>
Completed
┄┄┄┄┄┄┄┄┄
Enhancements
┈┈┈┈┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#601] Domain better participants
The `0(n_running_domains)' from domain creation and the iterations
`0(Max_domains)' from STW signalling have been removed.
• [ocaml-multicore/ocaml-multicore#605] Eventog event for condition
wait
A new event has been added to indicate when a domain is blocked at
`Condition.wait'. This is useful for debugging any imbalance in task
distribution in domainslib.
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/3/3576c6605c9fca962447ce44cd7a5f1c237b6c5e.png>
• [ocaml-multicore/ocaml-multicore#615] make depend
Updated `stdlib/.depend' to cover the recent developments in stdlib.
• [ocaml-multicore/ocaml-multicore#626] Add Obj.drop_continuation
Added a `caml_drop_continuation' primitive to `runtime/fiber.c' to
prevent leaks with leftover continuations.
[ocaml-multicore/ocaml-multicore#601]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/601>
[ocaml-multicore/ocaml-multicore#605]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/605>
[ocaml-multicore/ocaml-multicore#615]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/615>
[ocaml-multicore/ocaml-multicore#626]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/626>
Upstream
┈┈┈┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore#584] Modernise signal handling
The Multicore OCaml signals implementation is now closer to that of
upstream OCaml.
• [ocaml-multicore/ocaml-multicore#600] Expose a few more GC variables
in headers
The `caml_young_start', `caml_young_limit' and `caml_minor_heap_wsz'
variables have now been defined in the runtime.
• [ocaml-multicore/ocaml-multicore#612] Make intern and extern work
with Multicore
The upstream changes to intern and extern have now been incorporated
to work with the Multicore OCaml runtime.
[ocaml-multicore/ocaml-multicore#584]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/584>
[ocaml-multicore/ocaml-multicore#600]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/600>
[ocaml-multicore/ocaml-multicore#612]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/612>
Fixes
┈┈┈┈┈
• [ocaml-multicore/ocaml-multicore$604] Fix unguarded
`caml_skiplist_empty' in `caml_scan_global_young_roots'
A patch that fixes a locking bug with global roots observed on a Mac
OS CI with `parallel/join.ml'.
• [ocaml-multicore/ocaml-multicore#621] otherlibs:
`encode_terminal_status' does not set all fields
A minor fix for the error caused when moved from using
`caml_initialize_field' to `caml_initialize' in otherlibs.
• [ocaml-multicore/ocaml-multicore#628] In link_channel, channel->prev
should be set to NULL
A PR to fix the memory leak when using `flush_all' with `ocamlc' as
reported by Filip Koprivec.
• [ocaml-multicore/ocaml-multicore#629] Backtrace last exn is val unit
A fix for the crash reported on running core's test suite by
clearing `backtrace_last_exn' to `Val_unit' in
`runtime/backtrace.c'.
[ocaml-multicore/ocaml-multicore$604]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/604>
[ocaml-multicore/ocaml-multicore#621]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/621>
[ocaml-multicore/ocaml-multicore#628]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/628>
[ocaml-multicore/ocaml-multicore#629]
<https://github.com/ocaml-multicore/ocaml-multicore/pull/629>
Ecosystem
╌╌╌╌╌╌╌╌╌
Ongoing
┄┄┄┄┄┄┄
• [ocaml-multicore/ocaml-uring#36] Update to cstruct 6.0.1
ocaml-uring is now updated to use `Cstruct.shiftv' with the upgrade
to cstruct.6.0.1.
• [ocaml-multicore/domainslib#37] parallel_map
A request by @nbecker to provide a `parallel_map' function over
arrays having the following signature:
┌────
│ val parallel_map : Domainslib.Task.pool -> ('a -> 'b) -> 'a array -> 'b array
└────
• [ocaml-multicore/domainslib#38] parallel_scan rejects arrays not
larger than pool size
An "index out of bounds" exception is thrown for
`Task.parallel_scan' with arrays not larger than the pool size as
reported by @nbecker.
[ocaml-multicore/ocaml-uring#36]
<https://github.com/ocaml-multicore/ocaml-uring/pull/36>
[ocaml-multicore/domainslib#37]
<https://github.com/ocaml-multicore/domainslib/issues/37>
[ocaml-multicore/domainslib#38]
<https://github.com/ocaml-multicore/domainslib/issues/38>
Completed
┄┄┄┄┄┄┄┄┄
• [ocaml-multicore/eventlog-tools#4] Add `domain/condition_wait' event
The `lib/consts.ml' file in eventlog-tools now includes the
`domain/condition_wait' event.
• [ocaml-multicore/domainslib#34] Fix initial value accounting in
`parallel_for_reduce'
The initial value of `parallel_for_reduce' has been fixed so as to
not be accounted multiple times.
[ocaml-multicore/eventlog-tools#4]
<https://github.com/ocaml-multicore/eventlog-tools/pull/4>
[ocaml-multicore/domainslib#34]
<https://github.com/ocaml-multicore/domainslib/pull/34>
Eio
┈┈┈
The `eio' library provides an effects-based parallel IO stack for
Multicore OCaml.
◊ Ongoing
• [ocaml-multicore/eio#68] WIP: Add eio_luv backend
A work-in-progress to use `luv' that provides OCaml/Reason bindings
to libuv for a cross-platform backend for eio.
[ocaml-multicore/eio#68]
<https://github.com/ocaml-multicore/eio/pull/68>
◊ Completed
• [ocaml-multicore/eio#62] Update to latest MDX to fix exception
reporting
Dune has been updated to 2.9 along with necessary changes for
exception reporting with MDX.
• [ocaml-multicore/eio#63] Update README
A documentation update specifying the following steps required to
manually pin the effects version of `ppxlib' and
`ocaml-migrate-parsetree'.
┌────
│ opam switch create 4.12.0+domains+effects
│ --repositories=multicore=git+https://github.com/ocaml-multicore/multicore-opam.git,default
│ opam pin add -yn ppxlib 0.22.0+effect-syntax
│ opam pin add -yn ocaml-migrate-parsetree 2.1.0+effect-syntax
└────
• [ocaml-multicore/eio#64] Improvements to traceln
Enhancements to `traceln' to make it an Effect along with changes to
trace output and addition of tests.
• [ocaml-multicore/eio#65] Add Flow.read_methods for optimised reading
The addition of `read_methods' in the `Flow' module as a faster
alternative to reading into a buffer.
• [ocaml-multicore/eio#66] Allow cancelling waiting for a semaphore
Update to `lib_eio/semaphore.ml' to allow cancel waiting for a
semaphore.
• [ocaml-multicore/eio#67] Add more generic exceptions
The inclusion of generic exceptions to avoid depending on
backend-specific exceptions. The tests have also been updated.
[ocaml-multicore/eio#62]
<https://github.com/ocaml-multicore/eio/pull/62>
[ocaml-multicore/eio#63]
<https://github.com/ocaml-multicore/eio/pull/63>
[ocaml-multicore/eio#64]
<https://github.com/ocaml-multicore/eio/pull/64>
[ocaml-multicore/eio#65]
<https://github.com/ocaml-multicore/eio/pull/65>
[ocaml-multicore/eio#66]
<https://github.com/ocaml-multicore/eio/pull/66>
[ocaml-multicore/eio#67]
<https://github.com/ocaml-multicore/eio/pull/67>
Benchmarking
╌╌╌╌╌╌╌╌╌╌╌╌
Sandmark Nightly
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
Ongoing
┈┈┈┈┈┈┈
• [ocaml-bench/sandmark-nightly#4] Parallel notebook pausetimes
graphing for navajo results throws an error
The parallel Jupyter notebook for pausetimes throws a ValueError
that needs to be investigated.
• [ocaml-bench/sandmark-nightly#5] Status of disabled benchmarks
The `alt-ergo', `frama-c', and `js_of_ocaml' benchmark results that
were disabled from the Jupyter notebooks have to be tested with
recent versions of Multicore OCaml.
• [ocaml-bench/sandmark-nightly#6] Parallel scalability number on
navajo look odd
The parallel performance numbers on the navajo build server for
scalability will need to be reviewed and the experiments repeated
and validated.
• [ocaml-bench/sandmark-nightly#7] Use `col_wrap' as 3 instead of 5 in
the normalised results in parallel notebook
For better readability, it is recommended to use col_wrap as 3 in
the normalised results in the parallel notebook.
• [ocaml-bench/sandmark-nightly#8] View results for a set of
benchmarks in the nightly notebooks
A feature request to filter benchmarks by name or by tags when used
with Jupyter notebooks.
• [ocaml-bench/sandmark-nightly#9] Static HTML pages for the recent
results
The benchmark results from the most recent build runs should be used
to generate static HTML reports for review and analysis.
[ocaml-bench/sandmark-nightly#4]
<https://github.com/ocaml-bench/sandmark-nightly/issues/4>
[ocaml-bench/sandmark-nightly#5]
<https://github.com/ocaml-bench/sandmark-nightly/issues/5>
[ocaml-bench/sandmark-nightly#6]
<https://github.com/ocaml-bench/sandmark-nightly/issues/6>
[ocaml-bench/sandmark-nightly#7]
<https://github.com/ocaml-bench/sandmark-nightly/issues/7>
[ocaml-bench/sandmark-nightly#8]
<https://github.com/ocaml-bench/sandmark-nightly/issues/8>
[ocaml-bench/sandmark-nightly#9]
<https://github.com/ocaml-bench/sandmark-nightly/issues/9>
Completed
┈┈┈┈┈┈┈┈┈
• [ocaml-bench/sandmark-nightly#2] Timestamps are not sorted in the
parallel_nightly notebook
The listing of timestamps in the drop-down option is now sorted.
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/f/f3270c2d6da395e34fc32ebe3442036a74f104c3.png>
[ocaml-bench/sandmark-nightly#2]
<https://github.com/ocaml-bench/sandmark-nightly/issues/2>
Sandmark
┄┄┄┄┄┄┄┄
Ongoing
┈┈┈┈┈┈┈
• [ocaml-bench/sandmark#243] Add irmin tree benchmark
A request to add the Irmin tree.ml benchmark to Sandmark, including
necessary dependencies and data files.
• [ocaml-bench/sandmark#245] Add dune.2.9.0
An update to dune.2.9.0 in order to build coq with Multicore OCaml
on Sandmark.
• [ocaml-bench/sandmark#247] Sandmark breaks on OCaml 4.14.0+trunk
The Sandmark build for OCaml 4.14.0+trunk needs to be resolved as we
begin upstreaming more Multicore OCaml changes.
• [ocaml-bench/sandmark#248] coq fails to build
The `coq' package is failing to build with 4.12.0+domains+effects
with Sandmark on navajo server.
[ocaml-bench/sandmark#243]
<https://github.com/ocaml-bench/sandmark/issues/243>
[ocaml-bench/sandmark#245]
<https://github.com/ocaml-bench/sandmark/pull/245>
[ocaml-bench/sandmark#247]
<https://github.com/ocaml-bench/sandmark/issues/247>
[ocaml-bench/sandmark#248]
<https://github.com/ocaml-bench/sandmark/issues/248>
Completed
┈┈┈┈┈┈┈┈┈
• [ocaml-bench/sandmark#233] Update pausetimes_multicore to fit with
the latest Multicore changes
The Multicore pausetimes have now been updated for the 4.12.0
upstream and 4.12.0 branches which now use the new Common Trace
Format (CTF).
• [ocaml-bench/sandmark#235] Update selected benchmarks as a set for
baseline benchmark
You now have the option to only filter from the user selected
variants in the Jupyter notebooks.
<https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/7/7661e85fc50c9e1b4a6e7e76ca2929d8c4d3c19a.png>
• [ocaml-bench/sandmark#237] Run sandmark_nightly on a larger machine
The Sandmark nightly builds now run on a 64+ core machine to benefit
from the improvements to Domainslib.
• [ocaml-bench/sandmark#240] Add navajo specific parallel config.json
file
A navajo server-specific run_config.json file has been added to
Sandmark to run Multicore parallel benchmarks.
• [ocaml-bench/sandmark#242] Add commentary on grammatrix
A documentation update for the grammatrix benchmark on customised
task distribution via channels and the use of `parallel_for'.
• [ocaml-bench/sandmark#244] Add chrt to pausetimes_multicore wrapper
The use of `chrt -r 1' in paramwrapper is required with
`pausetimes_multicore' to use the taskset arguments.
• [ocaml-bench/sandmark#246] Add trunk build to CI
The .drone.yml file has now been updated to include 4.14.0+stock
trunk build for the CI.
[ocaml-bench/sandmark#233]
<https://github.com/ocaml-bench/sandmark/pull/233>
[ocaml-bench/sandmark#235]
<https://github.com/ocaml-bench/sandmark/issues/235>
[ocaml-bench/sandmark#237]
<https://github.com/ocaml-bench/sandmark/issues/237>
[ocaml-bench/sandmark#240]
<https://github.com/ocaml-bench/sandmark/pull/240>
[ocaml-bench/sandmark#242]
<https://github.com/ocaml-bench/sandmark/pull/242>
[ocaml-bench/sandmark#244]
<https://github.com/ocaml-bench/sandmark/pull/244>
[ocaml-bench/sandmark#246]
<https://github.com/ocaml-bench/sandmark/pull/246>
current-bench
┄┄┄┄┄┄┄┄┄┄┄┄┄
Ongoing
┈┈┈┈┈┈┈
• [ocurrent/current-bench#117] Read stderr from the docker container
We are able to run Sandmark-2.0 -alpha branch with current-bench
now, and it is useful to view the error output when running with
Docker containers.
• [ocurrent/current-bench#146] Replicate ocaml-bench-server setup
A request to dynamically pass the Sandmark benchmark target commands
to current-bench in order to create pipelines.
[ocurrent/current-bench#117]
<https://github.com/ocurrent/current-bench/issues/117>
[ocurrent/current-bench#146]
<https://github.com/ocurrent/current-bench/issues/146>
OCaml
╌╌╌╌╌
Completed
┄┄┄┄┄┄┄┄┄
• [ocaml/ocaml#10039] Safepoints
The PR has been cherry-picked on 4.13 and finally merged with
upstream OCaml.
We would like to thank all the OCaml users, developers and
contributors in the community for their valuable time and support to
the project. Stay safe and have a great summer if you are northern
hemispherically based!
[ocaml/ocaml#10039] <https://github.com/ocaml/ocaml/pull/10039>
Acronyms
╌╌╌╌╌╌╌╌
• AFL: American Fuzzy Lop
• CI: Continuous Integration
• CTF: Common Trace Format
• GC: Garbage Collector
• GCC: GNU Compiler Collection
• GTK: GIMP ToolKit
• HTML: HyperText Markup Language
• IO: Input/Output
• OPAM: OCaml Package Manager
• OS: Operating System
• PR: Pull Request
• STW: Stop The World
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] <mailto:alan.schmitt at polytechnique.org>
[the archive] <https://alan.petitepomme.net/cwn/>
[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>
[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>
[Alan Schmitt] <https://alan.petitepomme.net/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20210803/f648151d/attachment-0001.html>
More information about the caml-news-weekly
mailing list