firebird/pyrobird/README.md

0001 # pyrobird
0002
0003 [![PyPI - Version](https://img.shields.io/pypi/v/pyrobird.svg)](https://pypi.org/project/pyrobird)
0004 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pyrobird.svg)](https://pypi.org/project/pyrobird)
0005
0006 -----
0007
0008 ## Installation
0009
0010 ```bash
0011 pip install pyrobird
0012 ```
0013
0014 Optional dependencies:
0015
0016 - `batch` - install playwright, that allows to make screenshots in batch mode
0017 - `xrootd` - install libraries to read xrootd located files and URLs starting with `root://`
0018 - `dev` - install pytest and other development requirements, mainly to run tests
0019
0020 If using `batch` for screenshots, after installing playwright, you need to install browser binaries:
0021
0022 ```bash
0023 python -m playwright install chromium
0024 ```
0025
0026 If installed via pip, `xrootd` library requires compilation, so the system should have cmake,
0027 compiler and some xrootd dependencies installed.
0028
0029 For debian/ubuntu the packages to install to use xrootd:
0030
0031 ```bash
0032 sudo apt install build-essential libxrootd-client-dev cmake zlib1g-dev uuid-dev libssl-dev python3-dev
0033
0034 ```
0035
0036 Development installation
0037
0038 ```bash
0039 python -m pip install --editable .[dev,batch]
0040
0041 # with xrootd
0042 python -m pip install --editable .[dev,batch,xrootd]
0043 ```
0044
0045 Running with Gunicorn (development mode)
0046
0047 ```bash
0048 gunicorn --bind 0.0.0.0:5454 pyrobird.server:flask_app --log-level debug --capture-output
0049 ```
0050
0051
0052 ## Contributing
0053
0054 - [PEP8](https://peps.python.org/pep-0008/) is required
0055 - [Use Numpy style dockstring comments](https://numpydoc.readthedocs.io/en/latest/format.html)
0056 - [pytest](https://docs.pytest.org/en/latest/) is used for unit tests. Aim for comprehensive coverage of the new code.
0057 - Utilize [type hints](https://docs.python.org/3/library/typing.html) wherever is possible to enhance readability and reduce errors.
0058 - Use of specific exceptions for error handling is better. As described in the [Python documentation](https://docs.python.org/3/tutorial/errors.html) rather than general exceptions.
0059 - Contributions are subject to code review. Please submit pull requests (PRs) against the `main` branch for any contributions.
0060 - Manage dependencies appropriately. Add new dependencies to `pyproject.toml`. Provide a justification for new dependencies
0061
0062 ## Testing
0063
0064 To install dependencies with testing libraries included
0065
0066 ```bash
0067 pip install .[test]
0068 ```
0069
0070 Navigate to the pyrobird/tests and execute:
0071
0072 ```bash
0073 pytest
0074
0075 # To stop immediately on error and enter debugging mode
0076 pytest -x --pdb
0077 ```
0078
0079
0080 ## Development install
0081
0082 ```
0083 python -m pip install --upgrade --editable  .[dev]
0084 ```
0085
0086 # Pyrobird Server
0087
0088 This server allows Firebird to work with local files and the local file system as
0089 well as to complement frontend features such as opening XRootD files, etc.
0090
0091 Serve Pirobird locally and have access to files in current directory:
0092
0093 ```bash
0094 pyrobird serve
0095 ```
0096
0097 **pyrobird** (backend) allows **Firebird** (frontend) to access certain files on your system.
0098 For this reason pyrobird server has multiple endpoints such as `/api/v1/download`
0099 which allows to download files. There are library files served and by default Firebird
0100 has access to files in your current directory or a directory provided via `--work-path` flag
0101
0102
0103 #### **Available Options**
0104
0105 | Option                     | Short | Type    | Default | Description                                                                                                                                                                                                                                   |
0106 |----------------------------|-------|---------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
0107 | `--allow-any-file`         |       | Flag    | `False` | Allow unrestricted access to download files in the system. When enabled, the server allows downloads of all files which the running user has access to. **Use with caution**: It is considered dangerous in production environments.            |
0108 | `--allow-cors`             |       | Flag    | `False` | Enable CORS for downloaded files. This option should be used if you need to support web applications from different domains accessing the files, such as serving your server from a central Firebird server.                                       |
0109 | `--disable-files`          |       | Flag    | `False` | Disable all file downloads from the server. This option will prevent any file from being downloaded, enhancing security by restricting file access.                                                                                              |
0110 | `--work-path TEXT`         |       | String  | `CWD`   | Set the base directory path for file downloads. Defaults to the current working directory. Use this option to specify where the server should look for files when handling download requests.                                                       |
0111
0112
0113 > `--allow-any-file` - allows unrestricted access to download files in a system.
0114 > When enabled, the server allows downloads of all files that user has access to.
0115 > When disabled - only files in `--work-path` and its subdirectories are allowed to be downloaded.
0116 > This option could be considered safe on personal machines with a single user,
0117 > who runs localhost server in a terminal
0118 >
0119 > (!) It is considered dangerous in all other cases: farms, interactive nodes, production environments, servers, etc.
0120 > Just think `/etc/passwd` will be accessible through  `localhost:port/api/v1/download?f=/etc/passwd`
0121 >
0122 > So security wise, it is better to use `--work-path` than `--allow-any-file`
0123
0124
0125 - Start server with default settings, Firebird works with files in current directory:
0126    ```bash
0127   pyrobird serve
0128    ```
0129
0130 - Set where Firebird will take files from:
0131
0132    ```bash
0133    pyrobird serve --work-path=/home/username/datafiles
0134    ```
0135
0136    Now if you set file `local://filename.root` in Firebird UI,
0137    the file `/home/username/datafiles/filename.root` will be opened
0138
0139
0140 ## API Documentation
0141
0142 This is technical explanation of what is under the hood of the server part
0143
0144 ## Features
0145
0146 - **Secure File Downloading**: Download files with access control to prevent unauthorized access.
0147 - **EDM4eic Event Processing**: Extract and convert specific events from EDM4eic files to JSON.
0148 - **Static File Serving**: Serve frontend assets seamlessly alongside API endpoints.
0149 - **Dynamic Configuration**: Serve configuration files with real-time server information.
0150 - **CORS Support**: Enable Cross-Origin Resource Sharing for specified routes.
0151
0152 ### Configuration Options
0153 - **DOWNLOAD_PATH**: `str[getcwd()]`, Specifies the directory from which files can be downloaded when using relative paths.
0154 - **PYROBIRD_DOWNLOAD_IS_DISABLED**: `bool[False]` If set to `True`, all download functionalities are disabled.
0155 - **PYROBIRD_DOWNLOAD_IS_UNRESTRICTED**: `bool[False]`, allows unrestricted access to download any file, including sensitive ones.
0156 - **CORS_IS_ALLOWED**: `bool[False]`, If set to `True`, enables Cross-Origin Resource Sharing (CORS) for download routes.
0157
0158
0159
0160
0161 The API provides endpoints for downloading files, processing EDM4eic events, and serving configuration files. It also includes static file serving for frontend assets.
0162
0163 ---
0164
0165 ### Download File
0166
0167 #### **Endpoint**
0168
0169 ```
0170 GET /api/v1/download
0171 GET /api/v1/download/<path:filename>
0172 ```
0173
0174 #### **Description**
0175
0176 Allows users to download specified files. The download can be restricted based on configuration settings to prevent unauthorized access to sensitive files.
0177
0178 #### **Parameters**
0179
0180 - **Query Parameters**:
0181     - `filename` (optional): The name or path of the file to download.
0182     - `f` (optional): An alternative parameter for the filename.
0183
0184 - **Path Parameters**:
0185     - `filename` (optional): The path of the file to download.
0186
0187 **Note**: You can provide the filename either as a query parameter or as part of the URL path.
0188
0189 #### **Usage**
0190
0191 1. **Download via Query Parameter**
0192
0193    ```bash
0194    curl -O "http://localhost:5454/api/v1/download?filename=example.txt"
0195    ```
0196
0197 2. **Download via URL Path**
0198
0199    ```bash
0200    curl -O "http://localhost:5454/api/v1/download/example.txt"
0201    ```
0202
0203 #### **Security Considerations**
0204
0205 - **Access Control**: Ensure that `DOWNLOAD_ALLOW_UNRESTRICTED` is set appropriately to prevent unauthorized access.
0206 - **Path Traversal**: The server sanitizes file paths to prevent directory traversal attacks.
0207
0208 ---
0209
0210 ### Open EDM4eic Event
0211
0212 #### **Endpoint**
0213
0214 ```
0215 GET /api/v1/convert/edm4eic/<int:event_number>
0216 GET /api/v1/convert/edm4eic/<int:event_number>/<path:filename>
0217 ```
0218
0219 #### **Description**
0220
0221 Processes an EDM4eic file to extract a specific event and returns the event data in JSON format. Supports both local and remote files.
0222
0223 #### **Parameters**
0224
0225 - **Path Parameters**:
0226     - `event_number` (required): The number of the event to extract.
0227     - `filename` (optional): The path or URL of the EDM4eic file.
0228
0229 - **Query Parameters**:
0230     - `filename` (optional): The name or path of the file to process.
0231     - `f` (optional): An alternative parameter for the filename.
0232
0233 **Note**: You can provide the filename either as a query parameter or as part of the URL path.
0234
0235 #### **Usage**
0236
0237 1. **Process Local File via Query Parameter**
0238
0239    ```bash
0240    curl "http://localhost:5454/api/v1/convert/edm4eic/5?filename=path/to/file.edm4eic.root"
0241    ```
0242
0243 2. **Process Remote File via URL Path**
0244
0245    ```bash
0246    curl "http://localhost:5454/api/v1/convert/edm4eic/5/http://example.com/data/file.edm4eic.root"
0247    ```
0248
0249 ### Asset Configuration
0250
0251 #### **Endpoint**
0252
0253 ```
0254 GET /assets/config.jsonc
0255 ```
0256
0257 #### **Description**
0258
0259 Serves the asset configuration file (`config.jsonc`) with additional server information injected dynamically.
0260
0261 #### **Usage**
0262
0263 ```bash
0264 curl "http://localhost:5454/assets/config.jsonc"
0265 ```
0266
0267 ### Publishing
0268
0269 ```bash
0270 pip install --upgrade build twine
0271 python -m build && python -m twine upload dist/*
0272
0273 # You will have to setup your pip authentication key
0274 ```