[cwn] Attn: Development Editor, Latest OCaml Weekly News
Alan Schmitt
alan.schmitt at polytechnique.org
Tue Jun 2 00:49:02 PDT 2020
Hello
Here is the latest OCaml Weekly News, for the week of May 26 to June 02,
2020.
Table of Contents
─────────────────
Senior Software Engineer at Bloomberg L.P. in New York
opam-0install 0.1 - fast opam solver for CI, etc
The end of Camlp4
Camlp5 version 7.12 release (4.11 compatibility)
New Todo web app in js_of_ocaml
"not-ocamlfind": useful extensions to ocamlfind
zed, lambda-term and utop
v0.14 release of Jane Street packages
Multicore OCaml: May 2020 update
Other OCaml News
Old CWN
Senior Software Engineer at Bloomberg L.P. in New York
══════════════════════════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/senior-software-engineer-at-bloomberg-l-p-in-new-york/5850/1]
Vineet Ahuja announced
──────────────────────
Type of position - Full-time, candidate must be based around New York
or willing to relocate. This is not a remote position.
Bloomberg Derivatives Engineering is looking for software engineers
with 2 or more years of experience in OCaml, C++, Python, Javascript
or Scala. No prior knowledge of finance is required.
Derivatives are complex financial instruments which pose interesting
software engineering challenges, such as creating scalable distributed
systems or writing optimized algorithms when dealing with large
quantities of data.
A few recent OCaml projects include implementing The Elm Architecture
in OCaml to create declarative GUIs, implementing a financial DSL
built on top of OCaml and creating microservices that manage
derivatives contracts.
In addition, we are actively growing the community and usage of OCaml
within the company through general purpose projects, such as property
based testing, and tooling for generating XML schemas from OCaml
types. We have a monthly meetup and there are opportunities to attend
or present at conferences. Please apply at
([https://careers.bloomberg.com/job/detail/82641]).
Please feel free to reach out to me directly by email
(vahuja13 at bloomberg.net) if you have any questions. Thank you!
opam-0install 0.1 - fast opam solver for CI, etc
════════════════════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/ann-opam-0install-0-1-fast-opam-solver-for-ci-etc/5852/1]
Thomas Leonard announced
────────────────────────
I've just released [opam-0install] 0.1, a faster solver for opam
dependencies.
Opam's default solver is designed to maintain a set of packages over
time, minimising disruption when installing new programs and finding a
compromise solution across all packages (e.g. avoiding upgrading some
library to prevent uninstalling another program).
In many situations (e.g. a CI system building in a clean environment,
a project-local opam root, or a duniverse build) this is not
necessary, and we can get a solution much faster by using a different
algorithm.
opam-0install does that by using 0install's (pure OCaml) solver with
opam packages. It is currently being used in several places:
• I'm using it in [ocaml-ci] to find package combinations to test.
• @kit-ty-kate is using it in [http://check.ocamllabs.io/], replacing
the z3 solver.
• @avsm is using it in [duniverse].
Running `make test' runs some solves using the default opam solver and
the new 0install one, and compares the results. On my machine, I get:
[https://user-images.githubusercontent.com/554131/83002958-90a16b80-a005-11ea-9b47-1546406afa76.png]
I've only shown the first few tests, which are somewhat
cherry-picked. After that, it starts picking packages at random, and
the results aren't quite so biased ;-)
When 0install and opam find different solutions, it should always be
the case that 0install's solution is better for at least one component
(but it could be worse for several). Surprisingly, it seems that the
reverse is not true. In the screenshot above, opam's solution for
irmin.0.10.0 appears to be strictly worse than 0install's choice.
The most common reason for finding different solutions is to do with
jbuilder. The "best" version of jbuilder is `jbuilder.transition',
which depends on dune 1 (and conflicts with dune 2). 0install will
often pick this "best" version of jbuilder, which then forces dune < 2
and older versions of other packages. You can avoid this in your own
packages by listing your dune dependency before whatever depends on
jbuilder, since 0install optimises components in order. Of course, if
you require a package that needs dune 2 then that will also force the
issue.
It's also possible that I'm not using the opam API quite correctly in
the tests. When I use a version of opam-repository since [OCaml 4.10.0
release], opam likes to choose `ocaml-system.4.08.1', a package that
isn't even available. I guess I'm missing a call to some init function
somewhere.
[opam-0install] https://github.com/talex5/opam-0install-solver
[ocaml-ci] https://github.com/ocurrent/ocaml-ci/
[duniverse] https://github.com/ocamllabs/duniverse
[OCaml 4.10.0 release]
https://github.com/ocaml/opam-repository/commit/78830dd34212a35223ab7d57ff38c7cb67424ee6
The end of Camlp4
═════════════════
Archive: [https://discuss.ocaml.org/t/the-end-of-camlp4/4216/97]
ygrek announced
───────────────
camlp4 4.10 is now available in opam. Quoting the readme :
Later releases will try to keep camlp4 buildable, by
supporting new OCaml AST but not new syntax constructions,
which means camlp4 will be able to parse only OCaml
language up to 4.08. Rationale: existing code using camlp4
will still be buildable, but no new code should be written
with camlp4.
Camlp5 version 7.12 release (4.11 compatibility)
════════════════════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/ann-camlp5-version-7-12-release-4-11-compatibility/5872/1]
Chet Murthy announced
─────────────────────
New release 7.12 of Camlp5. Compatible with all OCaml versions >=
4.00.0, latest OCaml version 4.11+alpha1 included.
This is a compatibility release. The only significant change is that
we're trying to shift the documentation over to ReStructuredText (and
Sphinx). You can find that in the source-tree (`doc/rst') with
instructions for building it, though soon I'll move over a prebuilt
copy to [https://camlp5.github.io]. Critically, any new syntax
changes in ocaml 4.11.0 are not (yet) supported (but read below for
that).
Home page, including downloading and documentation at:
[https://camlp5.github.io/]
Enjoy!
N.B. This will probably be the last bugfix release in the 7.xx line;
the next release will be for Camlp5 version 8.00, which will have
massive changes, among them full support for PPX rewriters (and a
companion project with many PPX rewriters implemented using Camlp5).
This necessarily means support for all the new syntax thru Ocaml
4.11.0.
New Todo web app in js_of_ocaml
═══════════════════════════════
Archive:
[https://discuss.ocaml.org/t/new-todo-web-app-in-js-of-ocaml/5873/1]
Bikal Lem announced
───────────────────
I just finished a rewrite (up-to-date??) of Todo web app. It uses
js_of_ocaml, react(opam react), reactiveData, dune, ocaml 4.10.0 and
good old stdlib. Mostly, just good old standard tools that comes with
ocaml and js_of_ocaml compiler.
One aspect of the codebase is that I have tried to incorporate
component based implementation rather than the strict MVC
implementation.
I am announcing it here in the hope that those new to ocaml,
js_of_ocaml and in doing web app in ocaml can perhaps use it as
another specimen to their learning journey.
repo: [https://github.com/bikallem/jsoo_todomvc]
running demo: [https://bikallem.github.io/jsoo_todomvc/]
"not-ocamlfind": useful extensions to ocamlfind
═══════════════════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/not-ocamlfind-useful-extensions-to-ocamlfind/5885/1]
Chet Murthy announced
─────────────────────
This is "less than announcement", because this tool is so small, but I
thought I should write a little descriptive post about it anyway.
This post consists in two parts: 1. about debugging PPX rewriters, and
2. about building, Makefiles, and "composability". I guess they're
not connected, but this little tool doesn't seem worth two separate
posts, so ….. this post is to explain how to use a new opam package,
`not-ocamlfind'.
Two problems have bedeviled me (and maybe others) in using Ocaml for a
while now.
Debugging PPX Rewriters
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
When using or writing PPX rewriters, it's pretty bloody difficult to
dagnose errors, especially when using multiple PPX rewriters. I've
been writing my own using camlp5, and both when trying to figure out
what existing rewriters do (so I could reverse-engineer them) and
debugging my own, it was pretty painful. `ocamlc -dsource' really
isn't enough. So with `not-ocamlfind', you can take any `ocamlfind
ocamlc' invocation, viz.
┌────
│ ocamlfind ocamlc -package ppx_deriving.show simple_show.ml
└────
and replace `ocamlfind ocamlc' with `not-ocamlfind preprocess':
┌────
│ not-ocamlfind preprocess -package ppx_deriving.show simple_show.ml
└────
and get the preprocessed output (along with a list of the commands
used to produce that output), so you can run it yourself. In
addition, there's an executable `papr_official.exe' installed in the
`not-ocamlfind' package-directory, that you can invoke to convert text
to binary-AST and back. Notice how the steps are i. convert text to
binary AST; ii. run PPX rewriter; iii. convert binary AST to text.
Steps i and iii are of course not executed by `ocamlc': they're there
so we can run this outside of ocamlc and see the result.
┌────
│ ppx_execute: ocamlfind not-ocamlfind/papr_official.exe -binary-output -impl simple_show.ml /tmp/simple_showa44366
│ ppx_execute: /home/chetsky/Hack/Ocaml/GENERIC/4.10.0/lib/ppx_deriving/./ppx_deriving package:ppx_deriving.show
│ /tmp/simple_showa44366 /tmp/simple_show2225a0
│ format output file: ocamlfind not-ocamlfind/papr_official.exe -binary-input -impl /tmp/simple_show2225a0
└────
N.B. If you use this with camlp5, you need to add a "pr_o.cmo" (or the
equivalent" in order to get human-readable output.
That's all: just a way to more transparently debug PPX rewriter
invocations.
Makefiles and composable builds
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
When building moderate-to-large projects, none of the build-tools are
very suitable: a. Makefiles quickly grow unwieldy, esp. when .cmo
files from one directory are used as inputs in another; it gets
difficult to ensure minimal-work rebuilds ("change file f1 in
directory a, and everything downstream has to get rebuilt") while
still actually rebuilding what *does* need to be rebuilt.
b. ocamlbuild was …. impenetrable, and completely inimical to C/C++.
c. ditto Oasis. I used oasis a lot, and it was always difficult.
d. dune is pretty opaque too.
Of all these tools, Makefiles come closest, except for the complete
lack of "composability". So let's fix that!
a. Each subdirectory of a large build, as its last step, installs a
package into a "local install" repository, and other subdirectories
used those packages instead of directly referring to files from other
directories b. In the "depend" section of each subdirectory's
Makefile, it lists the package-directories it depends upon, thus:
┌────
│ EXTERNAL := $(shell $(OCAMLFIND) query -predicates byte $(PACKAGES))
│ $(CMO): $(EXTERNAL)
└────
So all the `.cmo' files in the directory get rebuilt, every time
any of the package-directories used in this build are changed. c. In
that last step, where the subdirectory's build installs an ocamlfind
package, the problem is that that install will update last-modified
times. So we need a new command:
┌────
│ $(NOT_OCAMLFIND) reinstall-if-diff pa_ppx_expect_test -destdir $(DESTDIR)/lib META $(TARGET) $(TARGET:.cma=.a)
│ $(TARGET:.cma=.cmxa) $(wildcard *.cmt*)
└────
`reinstall-if-diff' compares the files-to-be-installed, with the files
that are already in the package directory, and if there are any
differences, invokes `ocamlfind remove' followed by `ocamlfind
install'; if the files are identical, nothing is done. And we need a
"local-install" target:
┌────
│ local-install::
│ $(MAKE) DESTDIR=$(WD)/$(TOP)/local-install/ install
└────
With these changes, a large project's toplevel Makefile can simply
say:
┌────
│ all:
│ set -e; for i in $(SYSDIRS) $(TESTDIRS); do cd $$i; $(MAKE) all; cd ..; done
└────
and we can be assured that no superfluous build-steps will be run.
There *is* one little problem left, which I haven't cleaned-up, mostly
because it isn't that troublesome: if the directories in that list
`$(SYSDIRS)' aren't in topological order, then "make all" might fail.
But this is, I suspect, an easily solvable problem. Just not high on
the list of priorities right now.
All of the little snippets in this post are taken from the `pa_ppx'
project: [https://github.com/chetmurthy/pa_ppx]
and the `not-ocamlfind' project is available both from opam and on
github: [https://github.com/chetmurthy/not-ocamlfind]
zed, lambda-term and utop
═════════════════════════
Archive:
[https://discuss.ocaml.org/t/ann-zed-lambda-term-and-utop/5888/1]
ZAN DoYe announced
──────────────────
New releases of zed, lambda-term and utop are coming!
changes:
zed 3.1.0 (2020-05-30)
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
• Zed_edit:
• Set_pos action
• Insert_str action
and notable changes:
lambda-term 3.1.0 (2020-05-30)
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
• `LTerm_read_line' and `LTerm_vi':
• vi visual mode
• register support
utop 2.6.0 (2020-05-30)
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
• compatible with OCaml 4.11
• switch to the new parser exposed since 4.11
• Vi edit mode: register support
Below is an illustration, that I split and store parts of a phrase
into two registers(a and b) within the visual mode and then assemble
them together.
[https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/7/7603f61b1c56b418d880c48e55600ed9255d68f2.gif]
v0.14 release of Jane Street packages
═════════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/ann-v0-14-release-of-jane-street-packages/5893/1]
Xavier Clerc announced
──────────────────────
We are pleased to announce the v0.14 release of Jane Street packages!
This release comes with 15 new packages, and a number of fixes and
enhancements. The documentation for this release is available on our
website:
[https://ocaml.janestreet.com/ocaml-core/v0.14/doc/]
The remainder of this mail highlights the main changes since the v0.13
release; we hope it will be useful to developers in the process of
migrating to the new version. A comprehensive changelog is available
at the end.
Notable changes
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
• Updated `Bin_prot' functors `Binable.Of_sexpable' and
`Binable.Of_stringable' to take UUIDs.
• Changed `Base' and `Core' `partition*' functions from using `[ `Fst
of 'a | `Snd of 'b ]' to using `('a, 'b) Either.t'.
• Replaced `Date' functions `add_weekdays' and `add_business_days'
with functions that clarify the behavior w.r.t. weekends.
• Added syntax `while%bind', analogous to `if%bind'.
• Added to the `Async.Log.create' function a required unit argument,
and an optional argument to specify a `Synchronous_time_source.t'.
• Made `Core_kernel' deprecate the `Unix' module, recommending
`Core.Unix', with `Caml_unix' as a fallback.
• Changed `Core_kernel.failwiths' (aka `Error.failwiths') to require
its `~here' argument.
• Added `ppx_let' syntax `let%mapn' and `let%bindn', for n-ary map and
bind.
• Renamed `Expect_test_helpers' as `Expect_test_helpers_async', and
renamed `Expect_test_helpers_kernel' as `Expect_test_helpers_core'.
Removed the inclusion of `Expect_test_helpers_core' in
`Expect_test_helpers_async'.
• Increased Async's default maximum number of open file descriptors
from 2^15 to 2^16.
• Moved most of `Core.Iobuf' to a standalone library, `Iobuf', that
depends only on `Core_kernel', and can be used in javascript. Moved
the rest of `Core.Iobuf' into a standalone library, `Iobuf_unix',
that depends on `Core'.
New packages
╌╌╌╌╌╌╌╌╌╌╌╌
• `accessor' ([https://github.com/janestreet/accessor]): A library
that makes it nicer to work with nested functional data structures.
• `accessor_async' ([https://github.com/janestreet/accessor_async]):
Accessors for Async types, for use with the Accessor library.
• `accessor_base' ([https://github.com/janestreet/accessor_base]):
Accessors for Base types, for use with the Accessor library.
• `accessor_core' ([https://github.com/janestreet/accessor_core]):
Accessors for Core types, for use with the Accessor library.
• `expect_test_helpers_async'
([https://github.com/janestreet/expect_test_helpers_async]): Async
helpers for writing expectation tests.
• `expect_test_helpers_core'
([https://github.com/janestreet/expect_test_helpers_core]): Helpers
for writing expectation tests.
• `higher_kinded' ([https://github.com/janestreet/higher_kinded]): A
library with an encoding of higher kinded types in OCaml.
• `incr_dom_interactive'
([https://github.com/janestreet/incr_dom_interactive]): A monad for
composing chains of interactive UI elements.
• `incr_dom_sexp_form'
([https://github.com/janestreet/incr_dom_sexp_form]): A library for
building forms that allow the user to edit complicated types.
• `ppx_accessor' ([https://github.com/janestreet/ppx_accessor]):
[@@deriving] plugin to generate accessors for use with the Accessor
libraries.
• `ppx_fixed_literal'
([https://github.com/janestreet/ppx_fixed_literal]): Simpler
notation for fixed point literals.
• `ppx_log' ([https://github.com/janestreet/ppx_log]):
Ppx_sexp_message-like extension nodes for lazily rendering log
messages.
• `ppx_string' ([https://github.com/janestreet/ppx_string]): ppx
extension for string interpolation.
• `re2_stable' ([https://github.com/janestreet/re2_stable]):
Re2_stable adds an incomplete but stable serialization of Re2.
• `vcaml' ([https://github.com/janestreet/vcaml]): OCaml bindings for
the Neovim API.
Deprecations / Removals
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
`Async':
• Deleted functions that were deprecated in 2018 and earlier.
• Deprecated the `Deferred.choice' type alias, in favor of
`Deferred.Choice.t'.
• Removed `Pipe.init', which has been deprecated since 2016.
• Deprecated `Throttle.Deferred', which was a pointless alias for
`Deferred'.
`Core':
• Removed the `Iobuf_debug' module.
`Core_kernel':
• Deprecated the `Bug' exception.
• Made `Core_kernel' deprecate the `Unix' module, recommending
`Core.Unix', with `Caml_unix' as a fallback.
Moves
╌╌╌╌╌
`Core':
• Moved most of `Core.Iobuf' to a standalone library, `Iobuf', that
depends only on `Core_kernel', and can be used in javascript. Moved
the rest of `Core.Iobuf' into a standalone library, `Iobuf_unix',
that depends on `Core'.
• Moved the `Bigstring' module out of `Core' to a standalone library,
`Bigstring_unix'.
• Moved the `Linux_ext' module out of `Core' to a standalone library.
`Core_kernel':
• Renamed `Fqueue' functions to be consistent with `Queue' and
`Fdeque'.
• Moved `Bus' to a standalone library.
Changelog
╌╌╌╌╌╌╌╌╌
`Async':
• Deleted functions that were deprecated in 2018 and earlier.
`Async_kernel':
• Added `Pipe.folding_filter_map'' function.
• Added `Synchronous_time_source.next_alarm_fires_at' function.
• Added `Synchronous_time_source.Event.Option' module, an immediate
option with support for optional syntax.
• Added `Pipe.concat_pipe' function.
• Added `Scheduler.run_every_cycle_end', analogous to
`run_every_cycle_start'.
• Exposed `Scheduler.Expert.last_cycle_num_jobs' function.
• Added `Async_kernel_scheduler.Expert' functions
`run_every_cycle_{start,end}' and deprecated
`set_on_{start,end}_of_cycle'.
• Added `Async_kernel_scheduler.last_cycle_time' accessor function.
• Deprecated the `Deferred.choice' type alias, in favor of
`Deferred.Choice.t'.
• Exposed `Synchronous_time_source' function
`Expert.max_alarm_time_in_min_timing_wheel_interval'.
• Optimized `Time_source.advance_by_alarms' and
`Scheduler.yield_until_no_jobs_remain' to reduce the per-alarm cost
from `660ns to ~140ns. Added to ~yield_until_no_jobs_remain' an
optional argument, `?may_return_immediately:bool' to opt in to the
faster behavior.
• Fixed an issue with `Pipe.upstream_flushed', so that its result does
not become determined if an exception happens while processing some
of the elements.
• Removed `Pipe.init', which has been deprecated since 2016.
• Deprecated `Throttle.Deferred', which was a pointless alias for
`Deferred'.
`Async_rpc_kernel':
• Renamed `Pipe_rpc.Direct_stream_writer.Group.flushed' as
`flushed_or_closed', and changed it so that its result becomes
determined if the individual pipes are flushed *or* closed.
`Async_rpc':
• Added `Rpc.Connection.serve_inet' function, for synchronously
creating a TCP IP server.
`Async_unix':
• Added `Tcp.Server.create_sock_inet' function, which is synchronous
and serves only inet addresses.
• Added `Tcp.Server.create_inet' function, for synchronously creating
a TCP IP server.
• In `Unix' functions that create a `File_descr.t' supplied
`~close_on_exec:true' rather than calling `set_close_on_exec'.
• Changed blocking `Sexp.save*' functions from `@@deprecated' to
`@@alert'.
• Made `Async_unix' shadow `Core_kernel.eprint_s', which can block.
• Added to the `Log.create' function a required unit argument, and an
optional argument to specify a `Synchronous_time_source.t'.
• Made `Writer.flushed_*' functions consistently flush the synchronous
output channel if one is set.
• Fixed a bug in `Writer' error handling so that it calls
`stopped_permanently' when the writer is not allowed to write due to
file-descriptor flags.
• Added to the `Log' module the ability to set a `transform' function
applied to each message that is logged.
• Added `Unix' support for `flock', paralleling the existing support
for `lockf', and extended the `with_file' function to support both
mechanisms. Changed the `lockf' functions' lock names from `Read |
Write' to `Shared | Exclusive'.
• Added `Writer' functions `flushed_or_failed_*', which, unlike
`Writer.flushed', return a result when the underlying writer fails.
• Fixed a race condition in `Shutdown.don't_finish_before'.
• Added to `Writer.with_file' an optional `?syscall' argument.
• Improved `Fd' error messages for `syscall*' functions and
`with_file_descr_deferred_exn'.
• Improved `Writer.with_file_atomic''s error message when `f' closes
`t'.
• Increased Async's default maximum number of open file descriptors
from 2^15 to 2^16.
• Fixed a race in `Writer.with_file_atomic' and the `save*' functions
that use it that could cause the file permissions to be incorrect.
• Fix a bug in `Shutdown.shutdown_on_unhandled_exn' where a process
would never shutdown if `Debug.log' raised (e.g. if stderr was a
broken pipe).
• Added to `Sys.when_file_changes' an `on_exn' optional argument.
• Added `Process.send_signal' function that is safe against pid reuse.
• Added `Unix.waitpid_prompt' function, a version of `waitpid' that
guarantees that it's determined in the same async job where the
`wait' system call is done.
`Base':
• Fixed a bug in `Int.round_nearest', which could overflow.
• Added `Hashtbl.Merge_into_action.t' type.
• Eliminated the `Base_boot' library, `lib/base/boot', by including
its ppx expansions directly in the `Base' library, `lib/base/src'.
• Changed `partition*' functions from using `[ `Fst of 'a | `Snd of 'b
]' to using `('a, 'b) Either.t'.
• Added `Hashtbl' functions `find_and_call1' and `find_and_call2',
aimed at avoiding allocation.
• Improved the `Gc' module's deprecation message to recommend using
`Caml.Gc'.
• Added `List.is_suffix' function.
• Added `Map.combine_errors' function.
• Added `Result' functions `to_either' and `of_either', and deprecated
`ok_fst'.
• Added `Char.Caseless' submodule, analogous to `String.Caseless'.
• Added `String.Caseless' functions `is_substring*' and `substr*', for
case-insensitive substring matching.
• Added the `Nothing' module, moved from `Core_kernel'.
• Added `Ref' function `sets_temporarily', a generalization of
`set_temporarily' to multiple refs.
• Changed `Float.min' and `Float.max' to never return `Float.nan', and
instead to always return one of the arguments. This improves
performance by avoiding allocation.
• Added `Option.try_with_join'.
• Added `Int_intf.S' functions `clz' and `ctz', for leading and
trailing zeros in the binary representation of an int.
• Added `Nothing' module (moved from `Core_kernel'), which defines
`type t = |', the uninhabited type.
• Dropped support for OCaml < 4.07.
• Added `Source_code_position.of_pos', which allows you to create
`Source_code_position.t~s from the OCaml built-in ~Caml.__POS__'
special value.
• Added `Sequence' functions to convert between `Sequence.t' and
`Caml.Seq.t'.
• Added integer byte-swap functions, `bswap*'.
• Added `Set.are_disjoint' function.
• Added `String.Search_pattern' accessor functions `pattern' and
`case_sensitive'.
• Added `String' functions `chop_{suffix,prefix}_if_exists'.
• Added `Comparable' functors `Infix' and `Polymorphic_compare', which
implement the comparisons in their eponymous signatures, and require
only `compare' as input.
• Added `Blit.S1_distinct' interface, which allows the `src' and `dst'
types to differ.
• Added `Uniform_array.iteri' function.
• Added `Map.validatei' function.
`Bin_prot':
• Updated functors `Binable.Of_sexpable' and `Binable.Of_stringable'
to take UUIDs.
`Core':
• For `Unix' functions that create a `File_descr.t', added an optional
argument, `?close_on_exec:bool', for atomic creation of
close-on-exec file descriptors.
• Added `Unix' function `flock_blocking'.
• Improved `Unix.create_process_env''s error message when it is called
with `~prog_search_path:[]'.
• Removed the `Iobuf_debug' module.
• Added `Iobuf' functions `copy' and `clone'.
• Moved most of `Core.Iobuf' to a standalone library, `Iobuf', that
depends only on `Core_kernel', and can be used in javascript. Moved
the rest of `Core.Iobuf' into a standalone library, `Iobuf_unix',
that depends on `Core'.
• Added `Unix.Env' submodule, with type `Unix.Env.t = Unix.env', and
exposed functions for manipulating `Env.t'.
• Added `Filename' function `open_temp_file_fd', which is like
`open_temp_file' but returns a file descriptor rather than an out
channel.
• Made `Time_ns.{Span,Ofday}.Option' match `Quickcheck.S'.
• Moved the `Bigstring' module out of `Core' to a standalone library,
`Bigstring_unix'.
• Moved the `Linux_ext' module out of `Core' to a standalone library.
`Core_kernel':
• Added `Bus.Callback_arity.Arity5'.
• Generalized `Univ_map' to support a user-supplied key type.
• Added `Gc.For_testing' submodule, with functions for measuring
allocation, previously in `Expect_test_helpers_kernel'.
• Renamed `Fqueue' functions to be consistent with `Queue' and
`Fdeque'.
• Moved `Bus' to a standalone library.
• Added to `Option.Stable.V1.t' `@@deriving equal'.
• Added to `Time_ns.Stable.V1' `Map' and `Set' modules.
• Replaced `Date' functions `add_weekdays' and `add_business_days'
with functions that clarify the behavior w.r.t. weekends.
• Fixed a bug in `Command.run' that caused incorrect calls in
`let%test_unit' to mistakenly succeed.
• Added `Command.Auto_complete' submodule, naming the type of
`Arg_type.create''s `~complete' argument.
• Added `Date.Option' submodule, `Immediate_option.S'.
• Added submodule `Percent.Option :
Immediate_option.S_without_immediate'.
• Changed `Command.Param.choose_one' to disallow parameters which
don't have any CLI flags.
• Changed `Blang.or_' and `Blang.and_' to make trees that
short-circuit more quickly by traversing fewer `Blang.t' nodes at
`eval'.
• Added to `Command.run' an optional argument,
`?when_parsing_succeeds:(unit -> unit)', which runs before the
program's main thunk.
• Improved `Command.Param.choose_one''s error messages.
• Deprecated the `Bug' exception.
• Made `Core_kernel' deprecate the `Unix' module, recommending
`Core.Unix', with `Caml_unix' as a fallback.
• Added `Gc' function `allocated_words : unit -> int'.
• Changed `Core_kernel.failwiths' (aka `Error.failwiths') to require
its `~here' argument.
• Changed `Command' argument parsing so that `--' is only interpreted
as a prefix of exactly `--', an in particular not a prefix of
`--help'.
• Extended `String.Stable.V1' to match `Stringable.S'.
• Added `Hash_queue' function `lookup_and_remove'.
• Add `Set' support for writing `Set.V1.M(Elt).t' in stable types.
• Added `Byte_units.arg_type'.
• Extracted from `Command' the `Shape'-related code into a new file,
`command_shape.ml'.
• Made `Date.Option' match `Comparable.S_plain' and
`Quickcheckable.S'.
• Added `Quickcheckable.Of_quickcheckable*' functors.
• Added `Time_ns.Ofday.every' function.
• Made `[@@deriving quickcheck]' work with `Base'-style maps and and
sets, e.g. `type t = int Map.M(String).t [@@deriving quickcheck]'.
• Added to `Command' support for validating a command line, via the
function `Command_test_helpers.validate_command_line'.
• Added `Filename.to_absolute_exn' function.
• Updated `gc_stubs.c' to work better with OCaml 4.10.
• Exposed `Quickcheck.Shrinker.filter*' functions.
• Exposed the `Univ_map.Data' module type.
`Expect_test_helpers':
• Generalized `print_and_check_container_sexps' so that it applies to
stable container modules.
• Fixed the handling of quickcheck shrinkers to print only the final
failing value, rather than every value that was tried and failed.
• Added Async analogs of `Ref.set_temporarily' and `sets_temporarily':
`set_temporarily_async', `sets_temporarily_async'.
• Renamed `Expect_test_helpers' as `Expect_test_helpers_async', and
renamed `Expect_test_helpers_kernel' as `Expect_test_helpers_core'.
Removed the inclusion of `Expect_test_helpers_core' in
`Expect_test_helpers_async'.
`Expect_test_helpers_async':
• Removed `Expect_test_helpers_async.Expect_test_config'.
`Expect_test_helpers_base':
• Added functions `replace' and `replace_s', for rewriting strings and
sexps in output.
• Added function `hide_temp_files_in_string', for hiding temp file
names in expect-test output.
• Changed `require_sets_are_equal' to take a module matching
`Comparator.S' rather than a fragment of `Set.S', which better fits
with `Base' idioms.
`Incremental':
• Added function `node_value', to get the current value of a node,
even if it is stale or invalid.
• Added support for syntaxes `let%mapn' and `let%bindn'.
• Added `Incremental.Var.replace' function, analogous to
`Ref.replace'.
`Linux_ext':
• Added function `peer_credentials', for using the `SO_PEERCRED'
socket option.
`Stdio':
• Optimized `In_channel.input_all' to eliminate an unnecessary copy
between buffers.
`Timezone':
• Added `Timezone' support in javascript.
Anton Kochkov said
──────────────────
I noticed there is a [msgpack] library inside vcaml, maybe it makes
sense to update [http://opam.ocaml.org/packages/msgpack/] package?
[msgpack] https://github.com/janestreet/vcaml/tree/master/msgpack
Simon Cruanes replied
─────────────────────
It's a totally unrelated package, as far as I can tell? And there's
also `msgpck' on opam.
Ty Overby then said and Anil Madhavapeddy replied
─────────────────────────────────────────────────
The message-pack library inside of vcaml is a fully
functioning msgpack and msgpack-rpc implementation. We
decided to write a new library because the official
msgpack/msgpack-ocaml implementation doesn’t implement
Ext-types, which are required for Neovim, and the “msgpck”
package didn’t exist at the time.
This makes total sense. In opam now, there are three implementations
of message-pack:
• the "official" (i.e. first) one at
[https://github.com/msgpack/msgpack-ocaml], which has the extremely
cool property of being extracted from a Coq [specification], but
which has some [performance issues].
• The msgpck (note the missing `a') library which is a pure OCaml
implementation at [https://github.com/vbmithr/ocaml-msgpck], and
used as the driving library for ppx_protocol_conv_msgpack (note the
presence of the `a', but it uses `msgpck').
• And now the one in Base, which isn't exposed directly.
From the opam repository perspective, it would be nice to have these
merge into the upstream repository. We generally frown on contracted
names that duplicate larger ones, but I understand why the situation
arose in this case.
Next steps would be for the maintainers of all three to chat: who owns
the repo in the upstream, is it still maintained and used, and if not
what the procedure would be to replace it with one of the other
two. This might take some time, but that's ok; there's no rush. But
I'd rather not see a fourth competing implementation show up in a year
:slight_smile:
[specification]
https://github.com/msgpack/msgpack-ocaml/tree/master/proof
[performance issues] https://github.com/msgpack/msgpack-ocaml/issues/14
Yaron Minsky then corrected
───────────────────────────
• And now the one in Base, which isn't exposed directly.
Minor correction: it's part of Vcaml, not Base. (Though it does depend
on Base).
Multicore OCaml: May 2020 update
════════════════════════════════
Archive:
[https://discuss.ocaml.org/t/multicore-ocaml-may-2020-update/5898/1]
Anil Madhavapeddy announced
───────────────────────────
*Multicore OCaml: May 2020*
Welcome to the May 2020 update from the Multicore OCaml team! As with
[previous updates], many thanks to @shakthimaan and @kayceesrk for
help assembling this month's roundup.
A major milestone in May 2020 has been the completion of rebasing of
Multicore OCaml all the way from 4.06 to 4.10! The Parallel Minor GC
variant that performs stop-the-world parallel minor collection is the
[default branch] for the compiler, which means that compatibility with
C bindings is now much simpler than with the older minor GC design.
I've received many questions asking if this means that multicore OCaml
will "just work" with the opam ecosystem now. Not quite yet: we
estimate that we are now one PR away from this working, which requires
that the existing `Threads' module is backported to multicore OCaml to
support the older (non-parallel-in-the-runtime but concurrent) uses of
threading that existing OCaml supports. This effort was begun a year
ago by @jhw in [#240] and now rebased and being reviewed by @engil in
[#342]. Once that is merged and tested by us on a bunch of packages
and bulk builds, we should be good to start using Multicore OCaml with
opam. Stay tuned for more on that next month!
The ongoing and completed tasks for the Multicore OCaml are listed
first, which are then followed by improvements to the Sandmark
benchmarking project. Finally, the status of the contributions to
upstream OCaml are mentioned for your reference. This month has also
seen a meeting of the core OCaml runtime developers to assign
post-rebasing tasks (such as also porting statmemprof, how to handle
non-x86 architectures, Windows support, etc) to ensure a more complete
view of the upstreaming tasks ahead. The task list is long, but
steadily decreasing in length.
As to how to contribute currently, there is an incredibly exciting
seam of work that has now started on the appropriate programming
abstractions to support parallel algorithms in OCaml. See [this
thread] for more on that, and also on the [Domainslib] repository for
more low-level examples of traditional parallel algorithms. In a
month or so, we expect that the multicore switch will also be more
suitable for use with opam, but don't let that stop you from porting
your favourite parallel benchmark to Domainslib today.
[previous updates] https://discuss.ocaml.org/tag/multicore-monthly
[default branch]
https://github.com/ocaml-multicore/ocaml-multicore/tree/parallel_minor_gc
[#240] https://github.com/ocaml-multicore/ocaml-multicore/pull/240
[#342] https://github.com/ocaml-multicore/ocaml-multicore/pull/342
[this thread]
https://discuss.ocaml.org/t/language-abstractions-and-scheduling-techniques-for-efficient-execution-of-parallel-algorithms-on-multicore-hardware/5822/19
[Domainslib] https://github.com/ocaml-multicore/domainslib
Multicore OCaml
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌
◊ Ongoing
• [ocaml-multicore/ocaml-multicore#339] Proposal for domain-local
storage
A new proposal for implementing a domain-local storage in Multicore
OCaml has been created.
• [ocaml-multicore/domainslib#8] Task library slowdown if the number
of domains is greater than 8
This is an ongoing investigation on why there is a slowdown with
`domainslib' version 0.2 for the Game of Life benchmark when the
number of domains is greater than eight.
• [ocaml-multicore/ocaml-multicore#340] Fix Atomic.exchange in
concurrent_minor_gc
An implementation is provided for `Atomic.exchange' using
`Atomic.get' and `Atomic.compare_and_set' to obtain the correct
semantics to handle assertion failure in interp.c.
• [ocaml-multicore/ocaml-multicore#338] Introduce Lazy.try_force and
Lazy.try_force_val
The `Lazy.try_force' and `Lazy.try_force_val' functions are
implemented for concurrent lazy abstractions to handle the RacyLazy
exception.
• [ocaml-multicore/ocaml-multicore#333] Random module functions
slowdown on multiple cores
There is an observed slowdown for the `Random' module on multiple
cores, and the issue is being analysed in detail.
[https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/f/f5709f79b305f98fc24de9765606244572da0cbf_2_1380x428.png]
• [ocaml-multicore/ocaml-multicore#343] Fix extcall noalloc DWARF
The patch provides a fix for the emitted DWARF information for
`extcall noalloc'. This PR is currently under review.
[ocaml-multicore/ocaml-multicore#339]
https://github.com/ocaml-multicore/ocaml-multicore/issues/339
[ocaml-multicore/domainslib#8]
https://github.com/ocaml-multicore/domainslib/issues/8
[ocaml-multicore/ocaml-multicore#340]
https://github.com/ocaml-multicore/ocaml-multicore/pull/340
[ocaml-multicore/ocaml-multicore#338]
https://github.com/ocaml-multicore/ocaml-multicore/pull/338
[ocaml-multicore/ocaml-multicore#333]
https://github.com/ocaml-multicore/ocaml-multicore/issues/333
[ocaml-multicore/ocaml-multicore#343]
https://github.com/ocaml-multicore/ocaml-multicore/pull/343
◊ Completed
• [ocaml-multicore/ocaml-multicore#337] Update opam file to
4.10.0+multicore
The rebasing of Multicore OCaml to 4.11 branch
(`parallel_minor_gc_4_11') point is now complete! The [opam] file
for 4.10.0+multicore has been made the default in the multicore-opam
repository.
• [ocaml-multicore/ocaml-multicore#335] Add byte_domain_state.tbl to
install files
A patch to install `byte_domain_state.tbl' and `caml/*.h' files has
now been included in the runtime/Makefile which is required for
parallel_minor_gc_4_10 branch.
• The Multicore OCaml major GC implementation verification using the
SPIN model checker is available at the following GitHub repository
[ocaml-multicore/multicore-ocaml-verify].
[ocaml-multicore/ocaml-multicore#337]
https://github.com/ocaml-multicore/ocaml-multicore/pull/337/
[opam] https://github.com/ocaml-multicore/multicore-opam/pull/18/
[ocaml-multicore/ocaml-multicore#335]
https://github.com/ocaml-multicore/ocaml-multicore/pull/335
[ocaml-multicore/multicore-ocaml-verify]
https://github.com/kayceesrk/multicore-ocaml-verify
Benchmarking
╌╌╌╌╌╌╌╌╌╌╌╌
◊ Ongoing
• [ocaml-bench/sandmark#115] Task API Port: LU-Decomposition, Floyd
Warshall, Mandelbrot, N-body
Porting of the following programs - LU-Decomposition, Floyd
Warshall, Mandelbrot and N-body to use the Task API.
• [ocaml-bench/sandmark#37] Make benchmark wrapper user configurable
The ability to dynamically specify the input commands and their
respective arguments to the benchmark scripts is currently being
evaluated.
• [ocaml-bench/sandmark#106] Promote dune > 2.0
Sandmark works with dune 1.11.4 and we need to support dune greater
than 2.0 moving forward. The upgrade path with the necessary package
builds is being tested.
[ocaml-bench/sandmark#115]
https://github.com/ocaml-bench/sandmark/pull/115
[ocaml-bench/sandmark#37]
https://github.com/ocaml-bench/sandmark/issues/37
[ocaml-bench/sandmark#106]
https://github.com/ocaml-bench/sandmark/issues/106
◊ Completed
• [ocaml-bench/sandmark#109] Added sequential-interactive.ipynb
An interactive notebook to run and analyse sequential benchmarks has
been included. Given an artifacts directory with the benchmark
files, the notebook prompts you in the GUI to select different
commit and compiler variants for analysis. A sample screenshot of
the UI is shown below:
[https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/7/7399712c8843c2bde3626073bef857b180377ccf.png]
The PR adds error handling, user input validation and the project
README has also been updated.
• [ocaml-bench/sandmark#111] Add parallel initialisation and parallel
copy to LU decomposition benchmark
The parallel initialisation is now added to LU decomposition
numerical benchmark
(benchmarks/multicore-numerical/LU_decomposition_multicore.ml).
• [ocaml-bench/sandmark#113] Use –format=columns with pip3 list in
Makefile
A fix for the "DEPRECATION: The default format will switch to
columns in the future" warning when using `pip3 list' has now been
added to the Makefile with the use of the `--format=columns' option.
• [ocaml-bench/sandmark#116] Use sudo for parallel benchmark builds
The Makefile has been updated with the right combination of `sudo'
and OPAM environment variables so that we can now run parallel
benchmarks in Sandmark. The sudo command is required exclusively for
using the `chrt' command. We can now perform nightly builds for both
serial and parallel benchmarks!
• [ocaml-bench/sandmark#118] Refactored README and added JupyterHub
info
The Sandmark README file has now been updated to include information
on configuration, usage of JupyterHub, benchmarking and a quick
start guide!
[ocaml-bench/sandmark#109]
https://github.com/ocaml-bench/sandmark/pull/109
[ocaml-bench/sandmark#111]
https://github.com/ocaml-bench/sandmark/pull/111
[ocaml-bench/sandmark#113]
https://github.com/ocaml-bench/sandmark/pull/113
[ocaml-bench/sandmark#116]
https://github.com/ocaml-bench/sandmark/pull/116
[ocaml-bench/sandmark#118]
https://github.com/ocaml-bench/sandmark/pull/118
OCaml
╌╌╌╌╌
◊ Ongoing
• [ocaml/ocaml#9541] Add manual page for the instrumented runtime
A draft manual for the instrumented runtime eventlog tracing has
been created. Please feel free to review the document and share your
valuable feedback.
• [ocaml/dune#3500] Support building executables against OCaml 4.11
instrumented runtime
OCaml 4.11.0 has built-in support for the instrumented runtime, and
it will be useful to have dune generate instrumented targets.
[ocaml/ocaml#9541] https://github.com/ocaml/ocaml/pull/9541
[ocaml/dune#3500] https://github.com/ocaml/dune/issues/3500
◊ Completed
• [ocaml/ocaml#9082] Eventlog tracing system
The Eventlog tracing proposal for the OCaml runtime that uses the
Binary Trace Format (CTF) is now merged with upstream OCaml
(4.11.0).
• [ocaml/ocaml#9534] [RFC] Dynamic check for naked pointers
An RFC for adding the ability to dynamically identify naked pointers
in the 4.10.0 compiler.
• [ocaml/ocaml#9573] Reimplement Unix.create_process and related
functions without Unix.fork
The use of process creation functions in the Unix module is not
suitable for Multicore OCaml, for both behaviour and efficiency. The
patch provides an implementation that uses `posix_spawn'.
• [ocaml/ocaml#9564] Add a macro for out-of-heap block header
This PR adds a macro definition to construct a out-of-heap block
header in runtime/caml/mlvalues.h. The objective is to use the
header for out of heap objects.
As always, we would like to thank all the OCaml developers and users
for their continued support and contribution to the project. Stay safe
out there.
[ocaml/ocaml#9082] https://github.com/ocaml/ocaml/pull/9082
[ocaml/ocaml#9534] https://github.com/ocaml/ocaml/pull/9534
[ocaml/ocaml#9573] https://github.com/ocaml/ocaml/pull/9573
[ocaml/ocaml#9564] https://github.com/ocaml/ocaml/pull/9564
Acronyms
╌╌╌╌╌╌╌╌
• API: Application Programming Interface
• CTF: Common Trace Format
• DWARF: Debugging With Attributed Record Formats
• GC: Garbage Collector
• GUI: Graphical User Interface
• LU: Lower-Upper
• OPAM: OCaml Package Manager
• PIP: Pip Installs Python
• PR: Pull Request
• RFC: Request for Comments
• UI: User Interface
Other OCaml News
════════════════
>From the ocamlcore planet blog
──────────────────────────────
Here are links from many OCaml blogs aggregated at [OCaml Planet].
• [Every proof assistant: redtt]
[OCaml Planet] http://ocaml.org/community/planet/
[Every proof assistant: redtt]
http://math.andrej.com/2020/06/01/redtt-and-the-future-of-cartesian-cubical-type-theory/
Old CWN
═══════
If you happen to miss a CWN, you can [send me a message] and I'll mail
it to you, or go take a look at [the archive] or the [RSS feed of the
archives].
If you also wish to receive it every week by mail, you may subscribe
[online].
[Alan Schmitt]
[send me a message] mailto:alan.schmitt at polytechnique.org
[the archive] http://alan.petitepomme.net/cwn/
[RSS feed of the archives] http://alan.petitepomme.net/cwn/cwn.rss
[online] http://lists.idyll.org/listinfo/caml-news-weekly/
[Alan Schmitt] http://alan.petitepomme.net/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/caml-news-weekly/attachments/20200602/64526c3c/attachment-0001.html>
More information about the caml-news-weekly
mailing list