Reading and writting data

The SnowProfile object can be created by reading various data formats and can be saved in different formats. Functions to deals with input/output (i/o) of SnowProfile objects are gathered into the snowprofile.io module, which is documented below.

CAAML snowprofile format

Reading and writting of the CAAML international format for exchange of snow profile observation data is possible by the snowprofile package with the two following functions:

snowprofile.io.read_caaml6_xml(filename)[source]

Parse a CAAML 6 XML-based IACS Snow Profile document and return the associated SnowProfile object.

Versions up to 6.0.5 are currently supported.

This parser correctly deal with XML namespaces. One unique namespace URI have to be used for CAAML tags across the entire XML file for the parser to work correctly.

Most of the data stored in the CAAML format is parsed. However, some specific corner cases are not covered by this parser. Specifically, when dealing with profiles with unkown snow depth (not compulsoy in CAAML), the total depth may be set to 0 and the layer top/bottom height may be negative values.

Some data is partially parsed. For instance, missing numeric data could be specified as inapplicable (there is no value), missing (the value is not available when the data is written and may not exist), template (the value will be available later), unknown(the value is not available to the data writer but exists), or withheld (the value cannot be disclosed). All these cases are treated in the same way, using a python value None. Some parameters can be represented by text rather than numeric measurements (e.g. grain size could be reported in mm or by categories). To facilitate data processing, when categories are used, the package translates the category value into a numerical value refering to the center of the category.

Finally, CAAML may contain some additional data defined by the user. This cannot be parsed, as the structure is unknown. Most of this data could be stored and rewritten in a new file but not all, especially not those associated with layers in the different profiles.

Hence, reading a CAAML file and rewritten it with this package may cause some data loss. Most users, however, will experience no problems in this respect.

Parameters:

filename (str) – File path of the CAAML/XML document to parse

Returns:

The associated SnowProfile python object (or None in case of errors)

Return type:

SnowProfile

snowprofile.io.write_caaml6_xml(snowprofile, filename, version='6.0.5', indent=False)[source]

Write a SnowProfile object into a CAAML 6 XML-based IACS Snow Profile document.

Currently supported versions:

  • 6.0.6

  • 6.0.5 (default)

Parameters:

  • snowprofile (SnowProfile) – A SnowProfile object to dump to a CAAML file

  • filename (str) – The filename to write into. If already exists, will be overwritten.

  • indent (bool or string (spaces for indentation)) – Visually indent the output (default: False, provide the more compact outut available)

Internal JSON representation

Temporary storage of the data in an internal JSON-based format. Note that this format is not intended to be used for persistant data storage. Re-read of such data may or may not be compatible between versions of the package.

snowprofile.io.read_internal_json(filename)[source]

Read from an internal JSON representation to create a SnowProfile object.

Parameters:

  • filename (str or path-like object) – The filename/filepath to read.

  • kwargs – Arguments to be passed to the json.load function (standard library)

Returns:

The corresponding SnowProfile object

Return type:

:py:class:snowprofile.SnowProfile

snowprofile.io.write_internal_json(snowprofile, filename, **kwargs)[source]

Write the SnowProfile object into an internal JSON representation

Parameters:

  • snowprofile (:py:class:snowprofile.SnowProfile) – The SnowProfile object

  • filename (str or path-like object) – The filename/filepath to write (warning: any existing file with the same name will be overwritten with no confirmation).

  • kwargs – Arguments to be passed to the json.dump function (standard library)

Returns:

The written filename

Return type:

str

You can also get access (and re-read) to an internal JSON-based representation with:

snowprofile.io.to_json(snowprofile, **kwargs) → str[source]

Dump of a SnowProfile object into a JSON-encoded string

Parameters:

  • snowprofile (:py:class:snowprofile.SnowProfile) – The SnowProfile object to dump

  • kwargs – Arguments to be passed to the json.dumps function (standard library)

Returns:

JSON-serialized string

Return type:

str

snowprofile.io.from_json(json)[source]

Create a SnowProfile object from a JSON-encoded string.

Parameters:

json (str) – The JSON-encoded string

Returns:

The corresponding SnowProfile object

Return type:

:py:class:snowprofile.SnowProfile

Dict-based representation

You can get a dict-based JSON-serializable representation of a SnowProfile object with:

snowprofile.io.to_dict(snowprofile)[source]

Dump of a SnowProfile object into a JSON-serializable object (combination of dict, list and str).

Parameters:

snowprofile (:py:class:snowprofile.SnowProfile object) – The SnowProfile object to dump

Returns:

JSON-serializable object (dict)

Reading profiles from csv files

snowprofile.io.profile_csv.read_csv_profile(filename, mapper={}, profile_class=None, metadata={}, **kwargs)[source]

Read from a CSV file to create a SnowProfile object, containing the specified single profile.

The profile is as string refering to a classe defined in the snowprofile.profiles module, namely:

Example code for reading a density profile from a CSV file containing some comments at the begining (8 lines to skip), seperated by tabs.

from snowprofile.io.profile_csv import read_csv_profile

path_density = "densitydata_20240420.txt"

density_mapper = dict(
    top_height = dict(
        column = "Heigth[cm]",
        scale_factor = 0.01),
    thickness = dict(
        value = 0.025
    ),
    density = dict(
        column = "Density[kg/m3]",
        scale_factor = 1)
)

sp = read_csv_profile(
    filename=path_density,
    sep="\t",
    skiprows=8,
    mapper=density_mapper,
    metadata=dict(
        method_of_measurement="Snow Cutter"),
    profile_class="DensityProfile")

Each key of the mapper dictionary should map to a key in the data argument of the profile class, and the corresponding dictionary should provide:

  • column: the name of the column containing the data in the CSV file

  • scale_factor: a factor to multiply the values in the column (default=1)

  • value: a constant value to be used for all the rows of the profile (alternative to column)

metadata is a dictionary containing additional information to be added to the profile (it’s mandatory to provide the parameter and unit keys for ScalarProfile and VectorialProfile for example, and when it’s not mandatory, it can still be useful).

Parameters:

  • filename (str or path-like object) – The filename/filepath to read.

  • mapper (dict) – A dictionary mapping the keys of the profile to the columns of the CSV file.

  • profile_class (str) – The class of the profile to be created

  • metadata – Metadata to be added to the profile

  • kwargs – additional arguments to be passed to the pandas.read_csv function (standard library)

Returns:

The corresponding SnowProfile object

Return type:

:py:class:snowprofile.SnowProfile

Meteo-France internal database (Bdclim)

Readers to get operational observations from Meteo-France internal database structure are available (writting in this format is not possible).

A function allows to identify available observations:

snowprofile.io.search_mf_bdclim_dates(numposte, date_min, date_max=None, db_config={})[source]

Get the dates of observation for given num poste and between date_min and date_max.

See

Parameters:

  • numposte (str) – numposte

  • date_min (python datetime object or str) – begin date for search

  • date_max (python datetime object or str) – end date for search

  • db_config (dict) – The information to connect to the database

Returns:

list of available dates

Return type:

list of python datetime objects

A function allows to get the observation itself:

snowprofile.io.read_mf_bdclim(numposte, date, db_config={})[source]

Read snow profile from a database as stored in the Meteo-France climatological database. This routine is designed for internal use at Meteo-France only.

You have to provide database url and creedentials:

Keys to configure connexion are host, port, dbname, user, password.

Parameters:

  • numposte (str) – numposte

  • date (python datetime object or str) – date to read

  • db_config (dict) – The information to connect to the database

Returns:

The SnowProfile object containin the data

Return type:

SnowProfile