Dataset Viewer Tutorial

Introduction

Pan3D aids data scientists in exploring multidimensional datasets. For this tutorial, we will refer to a public multidimensional dataset provided by Pangeo Forge. This dataset was collected by the Copernicus Climate Change Service (C3S) as part of the Europe OBServational (E-OBS) gridded dataset. It contains observational data for precipitation, temperature, humidity, and air pressure.

For more information about this dataset, visit the C3S Catalog.

Get started

To follow along this tutorial, install Pan3D.

pip install pan3d[viewer]

Run the viewer as a local python server with the following command.

pan3d-viewer --dataset=https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/pangeo-forge/EOBS-feedstock/eobs-tg-tn-tx-rr-hu-pp.zarr

The Pan3D Viewer will open as a tab in your default browser. You can also visit localhost:8080 in another browser.

Note: to prevent the behavior of opening a tab on startup, add --server to the above command to run server mode.

Using the Viewer

Data configuration

After a moment to load the data from the remote URL, the Viewer will render the default configuration of the target dataset. This dataset contains daily weather data recorded over the European continent between years 1950 and 2020. When the dataset first loads, Pan3D displays the first array ("hu") at the first time step (01-January-1950).

You can open the left drawer by clicking on the dataset configuration icon in the top left.

Inside this panel, you will find the following information:

A group selection box. Its current value is "default". The only dataset in the default group is the one we passed as an argument. There is one more available group option in this selection box: the "xarray" group contains example Xarray datasets to explore. More groups can be added with the Catalog Search (See the Catalog Search tutorial for details).
A dataset selection box. This selection box contains all the datasets available in the currently selected group. Its current value is the url we passed as an argument, which is the only dataset available in the default group.
A button to view the attributes of the current dataset. Click the three-dots icon next to "Attributes" to open a dialog table of metadata available on the dataset.
A list of arrays available in the dataset, each with a button to view its attributes. The arrays in this dataset are acronymns, so we can open the attributes tables to see the standard names. The first array ("hu") is mean relative humidity, and we can see the unit is percentage.

You can select any array from this list by clicking on the name. For this tutorial, we will continue with the the mean temperature data in the array called "tg".

When a new data array is selected, the axis drawer on the right will open for further data configuration. This drawer allows us to change the default axis assignments and slicing. This drawer can be toggled with the axis info icon in the top right corner.

By default, "longitude" is assigned to X, "latitude" is assigned to Y, and "time" is assigned to T. This data does not have a Z coordinate, so our rendered meshes are all planes. We will explore a dataset with a Z axis later.

We can expand any of these coordinate panels. When we expand longitude, we see the following information

The attributes table of the coordinate. For longitude, the units are degrees east, and there are 705 values ranging from -24.95 to 45.45.
Inputs to adjust the slicing along the coordinate. For longitude, the default slicing starts at -24.95, stops at 45.45, and has a step of 1. This includes all values in the coordinate array.
A selection box to assign the coordinate to an axis. These coordinates have already been assigned to each axis automatically.

Note: Each time you change a value in this panel, Pan3D will attempt to make a new render. If you plan on making many changes before you want to re-render, disable the Auto Render feature with the checkbox in the top right. A button will appear when you have made changes that have not been applied. You can click this button to trigger the re-render once you have made all changes. The button displays the total size of the data that will be loaded for the render.

We can crop the rendered mesh and reduce its resolution by adjusting the slicing along these coordinates. After setting the start longitude to 0 and setting the step along longitude to 5, we get the following rendering.

We'll put our slicing back to how it was so we can see the full image again. (Set Start to -24.95 and Step 1.) Next, we'll expand the time coordinate and see that the panel is slightly different.

Instead of slicing options, the time coordinate has a slider for selecting one time step to display. Note that the time axis for this dataset has 25933 slices. Pan3D will only load the data for the current time step, so we don't load data that we don't need to render. This means that for each time change, Pan3D will fetch more data, but each fetch will be much faster than trying to load the whole dataset at once.

As noted above, the viewer displays the first time step (index 0) by default, which corresponds to 01-January-1950. We can see from the attributes table that the time coordinate range begins with this time and ends with 31-December-2020.

You can pick any index along this slider, and the label above will display the corresponding time step. Below, we have gone forward to 25-June-2004, and Europe appears much warmer.

Render configuration

Let's close the data configuration drawers and focus on the rendering area. There are many options to customize the appearance of the rendering within this space.

We can move the camera around the rendered mesh by clicking and dragging. We can pan the camera by holding Shift while dragging, and we can rotate it (roll) by holding Ctrl while dragging. We can move the camera toward the mesh and away from it by scrolling. We can see this mesh is a plane. We'll look at a 3D dataset soon.
The color legend is also interactive. We can drag it to another edge of the scene, or we can resize it by using the white bounding bars that appear when we click on the legend.
The circle with the three-dots icon in the top left opens a Views menu. Beside the three-dots icon, there are 12 buttons for you to try.

This menu contains the following options:
1. A button to reset the camera position and re-center the mesh.
2. A button to set the camera in a perspective view.
3. A button to put the camera on the X axis (our plane will be invisible from this view).
4. A button to put the camera on the Y axis (our plane will be invisible from this view).
5. A button to put the camera on the Z axis (this is the default view for our plane).
6. A button to toggle edge visibility (with our current high resolution, there are a lot of edges. Try zooming in when you enable this).
7. A button to toggle bounding box visibility (this will draw a thin border around our plane when enabled).
8. A button to toggle ruler visibility (these will show our latitude and longitude scales).
9. A button to toggle an axis widget's visibility (this will appear in the bottom left. Try rotating the scene while this is enabled).
10. A button to toggle between local and remote rendering mode. Local rendering is the default and is recommended for basic use cases.
11. A button to save the current visual as a static PNG file.
12. A button to save the current rendering as an interactive HTML scene.
The circle with the gear icon in the top right opens a rendering customization menu. This menu contains four customization options.
1. A colormap selection box. The default is "viridis". These options come from Matplotlib.
2. A checkbox to enable transparency. When enabled, another selection box will appear with options for transparency function. The default is "linear".
3. A checkbox to enable scalar warping. When enabled, scalar warping turns the rendered flat plane into a 3D mesh, where values are extruded in the Z axis according to their magnitudes.
4. Inputs to specify the relative scales of each axis. By default, this is a 1:1:1 ratio.

By using these configuration options, we can get a rendering like the one shown below. For this rendering, we did the following:

moved the color legend
changed the colormap to "plasma"
enabled transparency and changed the transparency function to "linear_r" (which means reverse linear)
enabled scalar warping
changed the axis scale ratio to 2:2:1 so the scalar warping would be less extreme
enabled ruler visibility
reset the camera to perspective view

Take a moment to try out different combinations to see how else this data can be configured to appear.

Saving configurations

The Pan3D viewer is intended to allow scientists to explore a dataset to find ideal visualizations with these many configuration options. Once you have found a visualization you like, you can use the PNG and HTML export options in the Views menu, but you can also export this configuration for fast replication within Pan3D.

After we finish our configuration and have a finalized rendering, we can click the "Export" button in the top toolbar. Clicking this button will open a dialog, which asks for a location to save a configuration file.

Some browsers, like Chrome, will allow specification of a download folder when you click on this input. Other browsers, like Firefox, do not allow this feature and will save the file to your default Downloads folder. By default, this file will be called pan3d_state.json. In browsers where you can specify download location, you can also change this name.

Once you have specified a download location, a file will be saved to your computer and the dialog will change.

For our configuration, the contents of the exported file appear as follows:

{
    "data_origin": "https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/pangeo-forge/EOBS-feedstock/eobs-tg-tn-tx-rr-hu-pp.zarr",
    "data_array": {
        "name": "tg",
        "x": "longitude",
        "y": "latitude",
        "t": "time",
        "t_index": 19899
    },
    "data_slices": {
        "time": [
            "Jan 01 1950 00:00",
            "Dec 31 2020 00:00",
            1
        ],
        "latitude": [
            25.05,
            71.45,
            2
        ],
        "longitude": [
            -24.95,
            45.45,
            2
        ]
    },
    "ui": {
        "loading": false,
        "main_drawer": false,
        "axis_drawer": false,
        "unapplied_changes": false,
        "error_message": null,
        "more_info_link": null,
        "expanded_coordinates": [],
        "current_time_string": "Jun 25 2004 00:00"
    },
    "render": {
        "auto": false,
        "x_scale": 2,
        "y_scale": 2,
        "z_scale": 1,
        "scalar_warp": true,
        "cartographic": false,
        "transparency": true,
        "transparency_function": "linear_r",
        "colormap": "plasma"
    }
}

To learn more about the schema to which this configuration file adheres, visit the Configuration Files documentation.

Now, a collaborator can easily replicate our rendering. Try to save the JSON contents above to a file on your computer and follow along as if you just received this file from a colleague.

Clicking the "Import" button in the top toolbar will open a similar dialog. Click the file input to select the location of the configuration file on your computer.

Once the file has been selected, another "Import" button will appear. After clicking this button, the dialog will change and Pan3D will begin loading the configuration and applying the changes.

After a moment to load, Pan3D will render the replicated scene.

These configuration files can be used as arguments in the local server startup command (see Command Line instructions for details) or can be used in a Jupyter notebook environment (see Jupyter Notebook tutorial for details).

Viewing other data

Open the left drawer again and select "xarray" in the Group selection box. The current dataset will be cleared and the Dataset selection box will be populated with seven example datasets from Xarray. You can look at any of these examples from Xarray and try out the configuration options we have reviewed.

There is one dataset among these with 4D data, which means we can select a time slice and get a 3D render. Select "Xarray Examples - ERA-Interim analysis" to experiment with these options on a 3D mesh.

This concludes the tutorial on how to use the Pan3D viewer.