A lot has happened since our last set of project updates. See all the recent updates from our projects below!
Sponsored Project Updates
Dask
- Dask support for pandas 2.2. Pandas 2.2 includes improvements that rely on the Apache Arrow ecosystem, like better PyArrow support. Dask is compatible with versions >= 2024.1.1. See the release notes.
- Dask with query optimization covers most of the Dask DataFrame API. Logical query planning for core Dask DataFrame is expected soon. More details in the project GitHub repo
- Partial rechunks for P2P rechunking. This benefits workloads requiring “local” rechunks, like rechunking squares to different squares. See the release notes.
New Blog Posts
- Real-world Grocery Demand Forecasting with XGBoost, Dask, and Coiled
- Xarray at Scale: A Beginner’s Guide
- Schedule Python Jobs with Prefect and Coiled
- One billion row challenge for Dask vs. Spark
Events
- Churning through cloud files in parallel. Thanks to those who joined the webinar yesterday! Watch the recording.
- Dask Demo Day: Apache Beam DaskRunner, expressions for Dask Array, and 1BRC for Dask vs Spark. Watch the recording.
New to Dask?
- Check out our pre-recorded tutorials on using Dask DataFrames, parallelizing your Python code, plus more advanced use cases.
Coiled
- Azure now works with Coiled. Reach out if you’re interested in becoming an early user.
- Schedule Python jobs with Prefect. New guide on how to deploy Prefect workflows on the cloud.
Just signed up for Coiled?
- Watch our short demo to see how to get started. It’s easy to connect Coiled to your AWS, GCP, or Azure account.
PyBaMM
- We have just released a new version: 24.1. We will publish the release notes on our website shortly.
- We are taking part in GSoC 2024 under the NumFOCUS umbrella.
napari
- napari just released version 0.4.19! This is primarily a bug fix release but it was a hard-fought one and fixes many long-standing bugs in addition to featuring some performance, API, and documentation improvements. See the Mastodon thread here and the full release notes here.
- napari received a targeted grant from the Chan Zuckerberg Imaging Institute to implement instanced rendering. We are looking for a paid short-term contractor to help with that effort, guided by napari and VisPy core developer Lorenzo Gaifas. If you think you might be interested, please get in touch by emailing napari-core-devs@googlegroups.com or by joining us on our Zulip chat room.
- Finally, if you have experience with polygon triangulation, we can use your help! Napari can’t currently display polygons with holes in them, which severely limits its utility for displaying geodata. That’s been a thorn in our side for some time now so if you have ideas about implementing this correctly we would super-appreciate the help! 😅👏
MDAnalysis
Releases
MDAnalysis v2.7.0 is now available on conda-forge and PyPi. A special thanks goes out to our community, especially to our 5 new contributors, as well as to NumFOCUS and the Chan Zuckerberg Initiative for their support!
Events
- MDAnalysis is now accepting abstracts for the 2024 MDAnalysis UGM, taking place August 21–23 in London, UK, at King’s College London, in partnership with the Thomas Young Centre.
- MDAnalysis, in collaboration with Molecular Nodes, has scheduled its next free online training workshop on February 28, 2024. Demand has been incredibly high, with more than 200 applicants signing up in the first 24 hours!
Additional Announcements
- We have started Phase II in our effort to relicense the MDAnalysis package from GPLv2+ to LGPLv2.1+ by reaching out to our ~200 contributors; see details in our September blog post.
- In an effort to modernize communications with our community, we have transitioned our user and developer mailing lists to GitHub Discussions; read more about this change on our blog.
- We had the opportunity to highlight the work of our GSoC 2023 students on our blog. We are also searching for more mentors for GSoC 2024 and encourage anyone interested to contact us on Discord or GitHub Discussions.
Blosc
- First public release of Caterva2, a new distributed system written in Python meant for sharing Blosc2 datasets among different hosts by using a publish–subscribe messaging pattern. Docs are over here: https://www.blosc.org/Caterva2/index.html. Feedback is welcome!
- New release of blosc2-grok plugin allowing to encode images with JPEG 2000. High-quality lossy compression arrives at Blosc2. We have blogged about it: https://www.blosc.org/posts/blosc2-grok-release/
- Maintenance releases for C-Blosc2 2.13.2 and Python-Blosc2 2.5.1. Among other things, the new versions allow the loading of Blosc2 plugins dynamically, without user intervention (other than installing the plugins. 🙂
Thanks to NumFOCUS for their continued support!
Spyder
- Spyder 5.5.1 is released, with new native M1 installers for macOS, support for formatting selections with Black, and a number of bug fixes.
- Spyder 6.0.0 Alpha 4 is now out, with a number of UI improvements, plotting enhancements, performance tweaks, and more over Alpha 3.
Affiliated Project Updates
Magpylib
Version 4.4 and 4.5 are out and focus mostly on interface and documentation improvements, field computation of triangular mesh bodies, and bugfixes
Optuna
We are working on the next minor release (v3.6) planned for March. We are planning to include several new features to Optuna v3.6, including:
- Wilcoxon pruner, a pruner based on Wilcoxon signed-rank test
- Native GP implementation, which supports mixed (with continuous, discrete and categorical) search spaces and is faster than BoTorchSampler
- PED-ANOVA importance evaluator
We are going to move the entire “optuna.integration” module to “optuna-integration” package. Please run “pip install optuna-integration” to use features under “optuna.integration”.
We are implementing a subset of Optuna features with Rust. This is useful for building Optuna bindings for other languages, such as C++ and Typescript.
pvlib
A lot has happened at pvlib since our last NumFOCUS update. The latest version, pvlib-0.10.3, was released in 2023–12–20. It had 19 contributors. Here are the highlights:
- Functions in pvlib.iotools for fetching solar resource data from SolarAnywhere and Solcast
- Forward and reverse irradiance transposition using the continuous Perez-Driesse models
- Parameter fitting and converter functions for popular IAM models
For the full list of what’s new and for highlights from past releases, please see the documentation.
Releases are available from PyPI and the conda-forge channel:
- https://pypi.org/project/pvlib/
- https://anaconda.org/conda-forge/pvlib and https://anaconda.org/conda-forge/pvlib-python
NOTE: new pvlib releases are no longer uploaded to the “pvlib” conda channel. Please install from PyPI or the conda-forge channel instead.
Read the Documentation: https://pvlib-python.readthedocs.io/en/stable/index.html
Report issues & contribute: https://github.com/pvlib/pvlib-python
Upcoming Events
The PV Performance Modeling Collaborative (PVPMC) will hold its annual workshop May 7–9, 2024, in Salt Lake City, UT. There will be talks, an update on pvlib, a user group meeting, and a tutorial. Registration details and the call for abstracts are here.
pyhf
We’re happy to announce that pyhf v0.7.6 is out on PyPI and Conda-forge! This is a small patch release with the following highlights:
- For the JAX backend, access jax.config from the jax top-level API to avoid support issues with jax and jaxlib v0.4.20+.
- Add information in the warnings for pyhf.infer.test_statistics.qmu and pyhf.infer.test_statistics.qmu_tilde that provides users with the higher level pyhf.infer APIs kwarg to set the correct test statistic.
- Correct the variable assignment for the one-sigma and two-sigma limit band artists in pyhf.contrib.viz.brazil.plot_brazil_band to match the stated return structure.
Please see the release notes for a full list of changes.
