About the workshop

Which file format should you use when saving your research dataset? Besides the obvious question of how to encode your data structures in a file, you might also want to consider portability (the ability to write/read data across different operating systems and different programming languages and libraries), the inclusion of metadata (data description), I/O bandwidth and file sizes, compression, and the ability to read/write data in parallel for large datasets. In this introductory in-person workshop, we will cover all these aspects starting from the very basic file formats and ending with scalable scientific dataset formats. We will cover CSV, JSON, YAML, XML, BSON, VTK, NetCDF, HDF5, using both structured and unstructured datasets and demoing all examples in Python.

Requirements

Device and software installation

• Bring your own laptop.
• In the demos we will use Python and quite a few Python libraries to write into all these file formats. If you are comfortable with running Python and installing Python libraries on your computer, we will mention the libraries as we go during the workshop. For everyone else, we will provide access to a remote system with all these libraries installed -- all you'll need is a browser.

Prerequisites
None.

Facilitators

Alex Razoumov

Alex Razoumov (SFU Research Computing /  Digital Research Alliance of Canada). Alex has a keen interest in difficult computational problems, with a PhD in computational astrophysics from the University of British Columbia and postdoctoral experience in Urbana–Champaign, San Diego, Oak Ridge, and Halifax. He has worked on numerical models ranging from galaxy formation to core-collapse supernovae and stellar hydrodynamics, and has developed a number of computational fluid dynamics and radiative transfer codes and techniques. Alex has been with Compute Canada in one role or another since 2009. He is based in Vancouver, British Columbia.

Marie-Hélène Burle

Marie-Helene Burle (SFU Research Computing / Digital Research Alliance of Canada). Prior to entering the realm of computing, Marie-Helene Burle spent 15 years roaming the globe from the High Arctic to uninhabited Sub-Antarctic islands or desert tropical atolls, conducting bird and mammal research (she calls those her "years running after penguins"). As a PhD candidate in behavioural and evolutionary biology at Simon Fraser University, she "fell" into Emacs, R, and Linux. This turned Marie into an advocate for open source tools and improved computing literacy for all, as well as better coding practices and more reproducible workflows in science. She started to contribute to the open source community, became a Software and Data Carpentry Instructor, and worked at the SFU Research Commons providing programming support to researchers. When not behind a computer, Marie loves reading history books and looking for powder in the British Columbia backcountry on skis.