November Project Updates

NumFOCUS

6 min readNov 27, 2024

Sponsored Projects

FEniCS

The FEniCS project has released v0.9.0, with many new features, see:

https://fenicsproject.org/blog/v0.9.0/ for details.

PyBaMM

The main update for now (though I don’t know if we already included it in the last one) is the first PyBaMM conference taking place next February in London: PyBaMM Battery Modelling Conference.

AstroPy

Astropy released version 7.0, in addition to new features. This version also includes numerous enhancements and bug fixes. Some of the new features improve compatibility within astropy sub packages and with other packages in the ecosystem, such as storing coordinate frames in tables, making astropy Quantity compatible with NumPy constructor functions, or improving compatibility with dask. Other features improve speeds or ease of use for existing astropy functionality.

Over 50 people have contributed code to this release; 20 of them for the first time. Thanks for all the community support!

Blosc

Announcing C-Blosc2 2.15.2: A fast, compressed and persistent binary data store library for C.

What is new?

This is a maintenance release in which we are fixing some issues that have been reported by the community. The most relevant changes are:

Support wasm32 by disabling the ZLIB WITH_OPTIM option. Thanks to Miles Granger.
Added support for nvcc (NVidia Cuda Compiler) in CMake. Thanks to @dqwu.
Fix public include directories for blosc2 targets. Thanks to Dmitry Mikushin.

For more info, please see the release notes at: https://github.com/Blosc/c-blosc2/blob/main/RELEASE_NOTES.md

What is it?

Blosc2 is a high-performance data container optimized for binary data. It builds on the shoulders of Blosc, the high-performance meta-compressor. Blosc2 is the next generation of Blosc, an `award-winning` library that has been around for more than a decade. Blosc2 expands the capabilities of Blosc by providing a higher lever container that is able to store many chunks on it (hence the super-block name). It supports storing data on both memory and disk using the same API. Also, it adds more compressors and filters.

Download sources

The GitHub repository is over here: https://github.com/Blosc/c-blosc2Blosc is distributed using the BSD license; see LICENSE.txt for details.

Mailing list

There is an official Blosc mailing list at: blosc@googlegroups.com

Twitter feed

Please follow @Blosc2 to get informed about the latest developments.

Enjoy Data!- The Blosc Development Team

Dask

Coiled referral program. Know someone who might have a good use case for Coiled? Receive up to $200 in Amazon gift cards when you refer someone to Coiled by Dec. 31st, 2024. Learn more.

SLURM-style job arrays on the cloud. HPC job scripts are simple and accessible to almost anyone. Oddly, they’re pretty hard to replicate on the cloud, so we replicated the API with Coiled. More in our blog post.

Better performance for Xarray GroupBy.Map. Running GroupBy-Map patterns backed by Dask arrays is common in large-scale geospatial workloads. The latest version of Dask uses a new algorithm for selecting data that’s more robust. Learn more in our blog post.

Legacy Dask DataFrame is being deprecated. Now is a great time to switch to the new DataFrame implementation which includes optimizations like column projection and filter pushdown. Please report any issues that come up. See the release notes.

Events

The next Dask Demo Day is on December 5th. Join us for live community demos and updates from Dask maintainers. Have something you’d like to share? Let us know.

PyData Global. Check out our talk “Dask Xarray: Geoscience at Massive Scale” on Thursday, Dec. 5th, at 4 pm UTC. Talk details here.

Just signed up for Coiled?

Watch our short demo to see how to get started. It’s easy to connect Coiled to your AWS, GCP, or Azure account.

New to Dask?

Check out our tutorials on using Dask DataFrames, parallelizing your Python code, plus more advanced use cases.

Affiliated Projects

skforecast

Skforecast 0.14.0 has just been released!

skforecast has undergone a major refactoring, improving both usability and speed, so you can forecast faster than ever!

Unified Forecaster Types

We’ve simplified the structure by unifying multiple 𝘧𝘰𝘳𝘦𝘤𝘢𝘴𝘵𝘦𝘳 types, reducing the complexity of classes while retaining all key functionalities.

Accelerated Multi-Series Forecasting

The 𝘍𝘰𝘳𝘦𝘤𝘢𝘴𝘵𝘦𝘳𝘙𝘦𝘤𝘶𝘳𝘴𝘪𝘷𝘦𝘔𝘶𝘭𝘵𝘪𝘚𝘦𝘳𝘪𝘦𝘴 is now considerably faster, making it possible to forecast at scale — even across thousands of series.

Window Features for All Forecasters

Now, all forecasters support window features, giving you more flexibility in feature engineering.

New cv Argument for Model Selection

The 𝘮𝘰𝘥𝘦𝘭_𝘴𝘦𝘭𝘦𝘤𝘵𝘪𝘰𝘯 functions now include a new 𝘤𝘷 argument. You can use 𝘛𝘪𝘮𝘦𝘚𝘦𝘳𝘪𝘦𝘴𝘍𝘰𝘭𝘥 or 𝘖𝘯𝘦𝘚𝘵𝘦𝘱𝘈𝘩𝘦𝘢𝘥𝘍𝘰𝘭𝘥 to define custom validation strategies

Seamless Migration with Simple Code Changes

Take advantage of these new features with just a few tweaks to your code! Check out our migration guide to get started smoothly.

With all these updates, we aim to make skforecast more user-friendly and aligned with the standards of scikit-learn, ensuring an intuitive experience for every user.

Release details: https://skforecast.org/0.14.0/releases/releases#0.14.0

Skforecast docs: https://skforecast.org

Thank you very much for helping to make the project’s development visible.

Mesa

Mesa is proud to announce the release of Mesa 3.0!

After our most extensive pre-release program ever. Mesa 3.0 brings major improvements to agent-based modeling, making it more intuitive and powerful while reducing complexity. This release modernizes core functionalities and introduces new capabilities for both beginners and advanced users.

TLDR: Check out a model | Get Started | See Examples

Streamlined agent management

The centerpiece of Mesa 3.0 is its new agent management system. Agents are now automatically tracked and assigned unique IDs, eliminating common boilerplate code. The new AgentSet functionality provides an elegant and flexible way to work with agents

The AgentSet provides powerful methods for filtering, grouping, and analyzing agents, making it easier to express complex model logic. Each model automatically maintains an AgentSet containing all agents (model.agents) and separate AgentSets for each agent type (model.agents_by_type). See AgentSet Examples

Modern Visualization with SolaraViz

Mesa 3.0’s new experimental visualization system, SolaraViz, provides a modern, interactive interface for model exploration:

Try the new interface at py.cafe

Note: SolaraViz is in active development. We might make API breaking changes between Mesa 3.0 and 3.1.

Enhanced data collection

The DataCollector now supports collecting different metrics for different agent types. Check it out

Experimental features

Mesa 3.0 introduces several experimental features for advanced modeling:

Cell Space with integrated PropertyLayers and improved agent movement capabilities
Voronoi grid implementation
Event-scheduling simulation capabilities

These experimental features are in active development and might break API between releases.

Transitioning from Mesa 2?

See our Mesa 3.0 migration guide for a full overview.

For questions or support, join our GitHub Discussions or Matrix Chat.

We would love to hear what you think about Mesa 3.0! Say hello here and leave any feedback on 3.0 here.

An incredible thanks to Ewout and Jan for leading this effort and doing the majority of the work!

PyVista

PyVista is a library for creating stunning visuals of 3D scientific data.

PyVista’s latest release was in July 2024 which focused on ensuring compatibility with NumPy 2.0, upgrades for future compatibility with VTK, enhanced documentation hosting and indexing by search engines. For a complete list of changes, visit the release notes and subsequent patch releases.

In upcoming work, we plan to enhance PyVista’s volume rendering capabilities by supporting dynamic visualization pipelines, unifying the volume rendering API with the robust mesh rendering capabilities, and providing comprehensive documentation. The improvements will optimize the performance and usability of volume rendering in PyVista, making it a more robust tool for scientific 3D visualization while addressing a backlog of bugs and feature requests.

A preview of PyVista’s latest developments (not yet released) include:

Using operators for boolean operations (see pull request)
result = sphere_a | sphere_b

Need help getting the most out of PyVista? Contact info@pyvista.org, and we can connect you with community experts or organizations with deep expertise in scientific computing.

For more detailed updates and to stay connected with the PyVista community:

GitHub Repository: https://github.com/pyvista/pyvista
Documentation: https://docs.pyvista.org
Discussions: https://github.com/pyvista/pyvista/discussions
Slack: https://slack.pyvista.org/

Optuna

Here are our updates for Optuna.

Optuna v4.1 is released with new features and performance improvements: https://github.com/optuna/optuna/releases/tag/v4.1.0
Optuna Dashboard v0.17.0 is released with a critical bug fix: https://github.com/optuna/optuna-dashboard/releases/tag/v0.17.0
Many new algorithms are added in OptunaHub!
AutoSampler: https://hub.optuna.org/samplers/auto_sampler/
SMAC3: https://hub.optuna.org/samplers/smac_sampler/
Hill Climb Local Search: https://hub.optuna.org/samplers/hill_climb_search/
Mulit-Objective CMA-ES: https://hub.optuna.org/samplers/mocma/
Multi-Armed Bandit Epsilon-Greedy Algorithm: https://hub.optuna.org/samplers/mab_epsilon_greedy/
NSGAII sampler with Initial Trials: https://hub.optuna.org/samplers/nsgaii_with_initial_trials/

We posted articles about new algorithms! Check them out: