What’s new with NumFOCUS projects: May 2024

A lot has happened since our last set of project updates. See all the recent updates below!

Sponsored Project Updates

Julia

JuliaCon 2024 is coming to the PSV Stadium in Eindhoven, Netherlands, between July 9th and 13th. It will be a day of workshops followed by three days of talks and poster presentations. The draft schedule of talks and workshops is now out. Tickets are still available; get yours today!

ITK

The Insight Toolkit (ITK) has introduced substantial enhancements in version 5.4 Release Candidate 4: ALL THE DICOMs, focusing on improved DICOM capabilities which are crucial for handling a wide range of medical imaging data. This update includes expanded support for various imaging modalities, particularly in processing and interpreting medical images, which is a significant step forward for ITK’s DICOM applications. With contributions from key community members and the implementation of modern C++ features, ITK continues to advance in usability and performance.

Key highlights from the ITK release:

• Enhanced DICOM support with expanded modality features and crucial spatial metadata for Secondary Capture images.
• Introduction of modern C++ support, enabling more efficient coding practices through features like structured bindings for multidimensional data.
• Improved Python support through Stable ABI Python wheels, facilitating better compatibility and future-proofing for newer Python versions.
• Encouragement for module developers to migrate to a scikit-build-core *pyproject.toml* file, enhancing the sustainability of ITK modules and support for mac ARM/Apple Silicon Python wheels.

The impact of spatial metadata handling on the NLM Visible Human cryomacrotome anatomic secondary capture images, available in the NIH Imaging Data Commons, when visualized in 3D Slicer.

Left: before ITK v5.4rc04, Right: after.

Screenshots courtesy of Steve Pieper.

Bokeh

The Bokeh Tutorial has been accepted for Scipy 2024. Details are https://cfp.scipy.org/2024/talk/JRLMLD/

mlpack

• mlpack was accepted into Google Summer of Code 2024 and has selected 6 students from 68 competitive proposals.
• mlpack won a NumFOCUS Small Development Grant to continue our work overhauling our documentation, creating application studies, and use case tutorials, and revamping our website.
• Work is ongoing and making great progress to integrate Bandicoot, a GPU linear algebra library, into mlpack.

Dask

Dask Updates

Efficient joins for Dask DataFrame. Dask DataFrame avoids unnecessary data shuffling, an expensive operation, for merges and groupby aggregations for Dask >= 2024.4.2. See the release notes.

New Blog Posts

• Example data pipeline with Prefect, Delta Lake, and Dask. Runnable example of a lightweight, scalable data pipeline that runs large Python jobs on a schedule. Read the blog post.
• How does Dask compare to Spark? In the latest TPC-H benchmark results, Dask is often faster than PySpark both locally and when processing a 10 TB dataset on the cloud. Read the blog post.

Events

Watch last month’s recording for demos on:

• Dask DataFrame is fast
• Large-scale population of vector databases for RAG
• Easy GPU access with Coiled

Have something you’d like to share? Let us know.

New to Dask?

Check out our pre-recorded tutorials on using Dask DataFrames, parallelizing your Python code, plus more advanced use cases.

Spyder

The first beta of Spyder 6 was released, which adds support for automatically connecting to and running code on remote hosts and more robust lockfile-based updates with our installers, along with all the new features introduced in previous alphas.

PyBaMM

• We held a training event at Oxford (UK) with over 50 attendees.
• We have 3 slots for the upcoming GSoC
• We got funding approved by The Faraday Institution to support the first PyBaMM conference. It will take place in the UK early next year, more details will follow in the next few weeks.
• Our website has now been improved, with a better and more accessible theme.

MDAnalysis

MDAnalysis, together with the Molecular Sciences Software Institute is holding a free, 2-day workshop on June 24-25, 2024, at Arizona State University in Tempe, Arizona, USA.

This workshop is intended to transform attendees from users to developers and will cover an introduction to MDAnalysis, software best practices (version control with git and GitHub, testing with pytest, documentation), and guide participants through the creation of their own “MDAKit”, a fully-fledged installable Python package that makes the participant’s code easily shareable within the research group, with collaborators, or the whole scientific community.

• Registration deadline May 27, 2024
• Register at https://www.mdanalysis.org/2024/05/14/ASUworkshop/#registration

Places are limited to 40 in-person participants; some sessions will be taught in hybrid mode.

A number of bursaries are available for travel and accommodation. Apply as part of registration.

The workshop is funded by the MolSSI (supported by the National Science Foundation), the Chan-Zuckerberg Initiative, and the Center for Biological Physics at Arizona State University.

• On May 10, 2024, MDAnalysis partnered with the Thomas Young Centre, the JC Maxwell Node of CECAM, and CCPBioSim to offer a free, hybrid workshop on an Introduction to Molecular Dynamics Trajectory Analysis using MDAnalysis. All workshop materials are publicly available on a GitHub repository, and the recording of the event is available on the MDAnalysis YouTube channel.
• Registration for the MDAnalysis UGM, taking place August 21–23, 2024 in London, UK, is now open to the public! Keep an eye on the UGM event page, the MDAnalysis blog, and MDAnalysis socials (LinkedIn, X, and Bluesky) to stay up-to-date on the event.
• The MDAnalysis team is excited to be hosting 3 Google Summer of Code and 1 Outreachy contributor this summer! The contributors introduced themselves and their projects on the MDAnalysis blog; make sure to check it out!

TARDIS

We’re excited to announce that the TARDIS team will be mentoring three new Google Summer of Code 2024 students this summer-

• Asish Kumar will be adding benchmarks to TARDIS.
• Sumit Gupta will be working on the packet tracking framework which will allow us to better investigate and process TARDIS simulation data.
• Sarthak Srivastava will implement the velocity packet tracker visualization tool, which will allow us to better understand the regions of the ejecta material that are most important for producing different SN features!

HoloViz

What is HoloViz? It is a suite of tools designed to simplify the process of creating interactive visualizations and dashboards, even for large and complex datasets.

Updates:

- Panel 1.4 was released in late March, followed by a few patch releases. Highlights include the addition of a dashboard builder interface that allows building a dashboard layout entirely using a drag-and-drop interface and the creation of tutorials. Check out the release blog post for more!

hvPlot 0.10 was released in early May. Check out the release blog post, which also highlights the features released in version 0.9, including integration with Polars, enhancements to the Explorer low-code interface, enhancements and documentation for large time series analysis, and an improved contributor experience.

Affiliated Project Updates

skforecast just released the new version 0.12.0

What’s new?

• Multiseries forecaster (𝐆𝐥𝐨𝐛𝐚𝐥 𝐌𝐨𝐝𝐞𝐥𝐬) can be trained using a series of different lengths and with different exogenous variables per series. See an example here: https://cienciadedatos.net/documentos/py44-multi-series-forecasting-skforecast.html
• 𝐁𝐚𝐲𝐞𝐬𝐢𝐚𝐧 𝐡𝐲𝐩𝐞𝐫𝐩𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫 𝐬𝐞𝐚𝐫𝐜𝐡 search is now available for all forecasters using Optuna. See an example here: https://skforecast.org/0.12.0/user_guides/hyperparameter-tuning-and-lags-selection#bayesian-search
• New functionality to select features using scikit-learn selectors. See an example here: https://skforecast.org/0.12.0/user_guides/feature-selection
• New ForecasterRnn to create forecasting models based on 𝐝𝐞𝐞𝐩 𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 (𝐑𝐍𝐍 𝐚𝐧𝐝 𝐋𝐒𝐓𝐌). See an example here: https://cienciadedatos.net/documentos/py54-forecasting-with-deep-learning.html
• New method to predict intervals conditioned on the range of the predicted values. This can help to improve the interval coverage when the residuals are not homoscedastic.
• All Recursive Forecasters are now able to differentiate the time series before modeling. See an example here: https://skforecast.org/0.12.0/faq/time-series-differentiation.html

Dive into the full details: https://skforecast.org

Happy forecasting!

ObsPy

We just did a bugfix release for ObsPy a few days ago, corresponding info:

ObsPy 1.4.1 was released. Anyone can find more information here.

Join our mailing list

Check out our user forum: https://discourse.obspy.org/

Follow us on X: https://twitter.com/obspy

OPTUNA

Optuna v3.6 is out with a lot of new features! Check out the release blog for more information. New features include:

• Wilcoxon pruner, a pruner based on Wilcoxon signed-rank test,
• Native GP implementation, which supports mixed (with continuous, discrete, and categorical) search spaces and is faster than BoTorchSampler
• PED-ANOVA importance evaluator is much quicker than the original f-ANOVA, and more.
• We have migrated the entire `optuna.integration` module to `optuna-integration` package. Please run `pip install optuna-integration` if you encounter an `ImportError`.
• We implemented a prototype of Rust version of Optuna. This significantly improves speed and is useful for building Optuna bindings for other languages, such as C++ and Typescript.
• We are working on the next major release (Optuna v4). We plan to stabilize many experimental features, remove deprecated ones, and implement new functionality.

Visual Python

• Visual Python is a GUI-based Python code generator developed on the Jupyter Lab, Jupyter Notebook, and Google Colab as an extension.
• Visual Python is an open-source project started for students who struggle with coding during Python classes for data science.
• If you would like to manage data with minimal coding skills, overcome learning barriers for Python, or eliminate repeated work using codes, try Visual Python.
• Supported features: Basic data analysis / Data pre-processing / Visualization / Machine learning / Statistics

Updates and Future issues

• We plan to release a new minor update(v3.0.2) very soon.
• We updated our documentation as a manual for Visual Python users. Visit our gitbook page to learn how to use Visual Python.
• Visit Visual Python Gitbook: https://visual-python.gitbook.io/docs
• We plan to support the VS Code extension in the future, and we always welcome new contributors. Please visit our Discord and GitHub communities.
• Visit Visual Python Discord: https://discord.gg/PypQrBZWZv
• Visit Visual Python Github: https://github.com/visualpython/visualpython

New to Visual Python?

• Visual Python is available on Jupyter Notebook, Jupyter Lab, and Colab. To get started, follow our instructions.
• Visit Visual Python homepage: https://visualpython.ai
• Are you not using any of them? Then, install our Visual Python Desktop. If you are a Windows user, you can just click Windows to try out Visual Python.

Crystal

The Crystal programming language is proud to announce the release of Crystal 1.12. The past two releases brought significant improvements, in particular, a great leap forward to supporting Windows. Also of relevance, we are working hard to improve the support for muli-threaded applications. Lastly, we have a new website! It includes great improvements in its visuals, accessibility, and access to information.

GNU Radio

• GNU Radio saw release 3.10.10.0 on the 22nd of April, and it *is* quite exciting! It ships with the new, Qt-based GNU Radio Companion (run `gnuradio-companion — qt` to use it), which is slated to be our graphical signal processing flow graph design tool for years to come. Find the detailed release notes in the Changelog.md, on the Github release page here, or in Josh’s project update video!

Upcoming Events

• The **European GNU Radio Days 2024** (EU GR days) take place in Darmstadt, Germany, from Aug. 27–30
• Focus this year: GNU Radio 4.0, modern DSP development in C++
• **GNU Radio Conference 2024** (GRCon’24) happens in Knoxville, Tennessee, USA, on Sept. 16–20
• Around 300 to 350 in-person participants
• Free main track online stream
• Topics range from GNU Radio core development over applications to radio science
• For information on both conferences, see https://events.gnuradio.org and the beginning of the project update video.

Calls for Participation

• For GRCon’24 and the EU GR days, you can find the CfPs on https://events.gnuradio.org
• GNU Radio is proud to partner with the International Amateur Radio Union to co-sponsor the Best Open-Source Experiment Award at the RADAR 2024 conference (Rennes, France). If you’re using FOSS tools (and seeing that you’re reading the NumFOCUS newsletter, you are!) and have done some interesting experiments involving Radar, MIMO, beamforming, and earth observation, your chances are good!

CVXPY

We released CVXPY 1.5, with many new features and other improvements:

• Switched the default solver for LPs and SOCPs from ECOS to Clarabel.
• Major updates to the documentation, adding a number of new sections to the User Guide and breaking up the monolithic Advanced features page
• Added .curvatures containing all curvatures an expression is compatible with
• Variable bounds can be specified with cp.Variable(bound=(lower, upper)) and are directly passed to the solver when helpful. lower and upper can be either a NumPy array or floating point number.
• Constants can be named by writing cp.Constant(name=’…’)
• Added a new atom, vdot, that has the same behavior as scalar_product
• CVXPY runs in the next PyOdide release via wasm
• Added or-tools 9.9 support
• Major rewrite to the PDLP interface
• Dropped MOSEK <= 9 support and upgraded the MOSEK integration code

Yellowbricks

Yellowbrick plans to resolve two open issues that we have regarding our Cook distance visualizer and to fix a bug with our classification report visualizer. We also plan to work on our open PRs and issues.

pvlib python

Dear NumFOCUS friends,

The pvlib python maintainers are happy to release v0.10.5. This release contains contributions from 9 people!

v0.10.5 is a bugfix release. Notably, it should be compatible with the upcoming numpy 2.0 release. It also drops support for Python 3.7. For the full list of what’s new, see the documentation: https://pvlib-python.readthedocs.io/en/stable/whatsnew.html

Google Summer of Code:

pvlib is proud to participate in GSoC under the NumFOCUS umbrella for the second time. This year pvlib will be mentoring three students, Echedey Luis Alvarez, Ioannis Sifnaios, and Rajiv Daxini under mentorship from maintainers Kevin Anderson and Adam R. Jensen. The students will contribute to a wide range of areas including spectral and electrical mismatch, shading, floating PV, and agrivoltaic irradiance models.

Releases are available from PyPI and the conda-forge channel:

• https://pypi.org/project/pvlib/
• https://anaconda.org/conda-forge/pvlib and https://anaconda.org/conda-forge/pvlib-python
• NOTE: new pvlib releases are no longer uploaded to the “pvlib” conda channel. Please install from PyPI or the conda-forge channel instead.

Read the Documentation:

• https://pvlib-python.readthedocs.io/en/stable/index.html

Report issues & contribute:

• https://github.com/pvlib/pvlib-python

Thank you for using pvlib python!

Mesa

Mesa is pleased to announce the release of Mesa 2.3.0.

Release Overview:

Mesa 2.3.0 is a big feature release and the last feature release before 3.0.

There are two main new features:

The experimental cell-centric discrete spaces, added in #1994, allow cells with not only properties but also active behaviors: the CellAgent. It's inspired by NetLogo’s patches but extends and this concept further.

We now have full support for discrete event scheduling, as added in #2066. It allows scheduling events (like Agent actions) at any time, including non-integer timesteps.

There are many other features: The Jupyter visualization now supports an easier way to specify sliders, NetworkGrid.get_neighbors() supports a radius, AgentSet.get() can retrieve multiple attributes, and there are now benchmarks to track Mesa performance during development.

Finally, 2.3.0 stabilizes the AgentSet (including model.agents), making it the first experimental Mesa feature that is taken out of its experimental phase.

Three Google Summer of Code (GSoC) contributors will join Mesa in Summer 2024

We are humbled by the overwhelming response to our first year being selected for GSoC, with an impressive 51 submissions to Mesa. It was a challenging task to review so many well-written proposals and code contributions.

We are excited to announce that three contributors will join us through Google Summer of Code (GSoC) to advance three efforts for Mesa.

These efforts include increasing its performance and scalability, enabling better functionality and decision-making, and better integrating demand machine learning functionality of reinforcement learning into the Mesa ecosystem.

The individuals joining us and the projects they will be working on are listed below.

Adam Amer is working on mesa-frames: Vectorized Operations for Performance and Scalability.

Dong Jun is working on Cacheable Mesa.

Harsh Mahesheka is working on Mesa RL.

We eagerly anticipate sharing the progress and updates as development unfolds. We welcome and encourage all those who submitted proposals to stay involved with Mesa, we certainly wish we were allowed to accept more.