.. title:: Docs .. image:: _images/logo.png :target: index.html :width: 40% .. raw:: html
.. image:: http://img.shields.io/badge/License-BSD--3--Clause--LBNL-blue.svg :target: https://gitlab.com/ml4science/mldas/-/raw/master/LICENSE.txt .. image:: https://badge.fury.io/py/mldas.svg :target: https://pypi.python.org/pypi/mldas/ .. image:: https://img.shields.io/pypi/dm/mldas .. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4132521.svg :target: https://doi.org/10.5281/zenodo.4132521 The Machine Learning for Distributed Acoustic Sensing data package (or MLDAS) was developed at `Lawrence Berkeley National Laboratory `_ and in close collaboration with geophysicists with the aim to allow intense analysis on large data scale, thereby significantly enhancing the current framework traditionally used within the community. While many extra features are available for quick exploratory data analysis, the core of this package relies on a fully parallelized set of machine learning routines powered by the `PyTorch `_ library. This documentation not only aims at demonstrating how the MLDAS software can be used on DAS data, it also provides useful Jupyter notebook examples which can be followed and executed through Google Colaboratory. Installation ------------ The package can be easily installed via the ``pip`` Python package manager as follows:: sudo pip install mldas In order to update the software when new releases come out, the following command can be used:: sudo pip -U install mldas Dependencies ------------ The following 8 dependent libraries are required in order to install the package: ``h5py``, ``hdf5storage``, ``matplotlib``, ``mpi4py``, ``numpy``, ``pillow``, ``scipy``, ``torch``, ``torchvision``, and ``yaml``. All those packages will be automatically installed during the ``pip`` installation. Package structure ----------------- A directory tree of this package is shown below. The Python routines present at the top level (i.e. ``assess.py``, ``gather.py``, ``parse.py``, ``prepare.py``, ``selection.py`` and ``train.py``) can all be run as executable while the rest of the modules are placed within specific sub-directory, each of them serving a particular purpose. .. code-block:: bash ├── __init__.py ├── assess.py ├── datasets │   ├── __init__.py │   ├── dataset.py │   └── util.py ├── explore │   ├── __init__.py │   ├── colab.py │   ├── distrib.py │   ├── evaluation.py │   ├── loading.py │   ├── mapping.py │   ├── prepare.py │   ├── single.py │   └── success.py ├── gather.py ├── models │   ├── __init__.py │   └── resnet.py ├── parse.py ├── prepare.py ├── selection.py ├── train.py ├── trainers │   ├── __init__.py │   ├── base.py │   ├── gan.py │   └── generic.py └── utils ├── __init__.py ├── distributed.py └── logging.py License Agreement ----------------- .. code-block:: html Machine Learning for Distributed Acoustic Sensing data (MLDAS) Copyright (c) 2020, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: (1) Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. (2) Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. (3) Neither the name of the University of California, Lawrence Berkeley National Laboratory, U.S. Dept. of Energy nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. You are under no obligation whatsoever to provide any bug fixes, patches, or upgrades to the features, functionality or performance of the source code ("Enhancements") to anyone; however, if you choose to make your Enhancements available either publicly, or directly to Lawrence Berkeley National Laboratory, without imposing a separate written license agreement for such Enhancements, then you hereby grant the following license: a non-exclusive, royalty-free perpetual license to install, use, modify, prepare derivative works, incorporate into other computer software, distribute, and sublicense such enhancements or derivative works thereof, in binary and source code form. .. toctree:: :caption: Software Structure :hidden: structure/executables structure/configs structure/trainers structure/production structure/explore .. toctree:: :caption: Platform Flexibility :hidden: platform/colab platform/hpc platform/wrapper platform/train .. toctree:: :caption: Exploratory Data Analysis :hidden: exploration/save_load.ipynb exploration/filtering.ipynb exploration/xcorr.ipynb exploration/stack.ipynb .. toctree:: :caption: Machine Learning Applications :hidden: clustering/ae clustering/vae clustering/tsne clustering/umap classification/probmap.ipynb segmentation/segment.ipynb .. toctree:: :caption: Published Works :hidden: papers/surface_wave_identification.ipynb papers/grid_hyperparameter_search.ipynb