Commit bafa74c0 authored by Jan Pöppel's avatar Jan Pöppel
Browse files

added docs and setup/testscripts

parent 893b4b26
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# JS Build-files
*.map
/node_modules
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Flask stuff:
instance/
.webassets-cache
# pyenv
.python-version
# dotenv
.env
# virtualenv
venv/
ENV/
recursive-include webviz/templates *
recursive-include webviz/static *
\ No newline at end of file
# webViz
# WebViz
A HTML+JS based visualization tool capable of communicating via various middlewares
\ No newline at end of file
A HTML+JS based visualization tool capable of communicating via various middlewares.
A Python backend establishes connection specified in the HTML frontend and passes JSON messages received over these connections to the
frontend. The frontend provides different modules which can react to incoming data, presenting it in various ways, ranging from simple plots (lines and bars), control elements such as buttons or sliders all the way up to complex visualizations.
![Example GUI](docs/img/GUI.png)
For some information about the history, the ideas and some technical aspects behind this tool, have a look at [this presentation](docs/webVizBackground.pdf)
## Installation
If you want to install the tool for general use in the system/environment, you can just use ``python setup.py install``. This will install the ``webviz.py`` as an "executable" called ``webviz``.
## Usage
WebViz is a standalone visualization tool, in principle it does not make any assumptions about how or where the data you want to visualize is generated. It only currently expects any data to be interpretable as JSON messages. Therefore, as long as your generated data is packaged into a JSON message, it can be received by WebViz. The available components each will make their own, more or less strict assumptions regarding the structure of the data (e.g. the LinePlot module will expect numeric data, either as complete lists, or singular values, which will be accumulated by the frontend). For more information regarding the different components refer to the [component documentation](#available-components) below.
In order to start the backend, you just need to run the ``webviz`` script, either by calling it directly in case you have installed it, or running ``python webviz.py`` from the directory containing the source code. This should directly open you browser and navigate to ``localhost:5000``, where you should see the empty dashboard:
![Dashboard Controls](docs/img/dashboardCtrl.png)
In the modules tab, you can select what module you want to add which will bring up a configuration dialog specific for that module:
![LinePlot Config](docs/img/linePlotConfig.png)
The mist important aspects to configure are the data sources. These sources consist of a ``channel`` and a ``dataKey``. The channel specifies the protocol which transports the data (currently ``tcp, ipaaca, rsb`` or ``zmq``) followed by a ``:`` and finally the "address" within the protocol: The port for tcp/zmq connections or the topic/channel in case of ipaaca and rsb.
All messages received are expected to contain JSON-formatted data. The dataKey specifies which key within this JSON-object should be considered and whose values will be added to the store.
## Requirements and development setup
When you want to develop new components/modules for the visualization, you will need to setup the appropriate development environment for both the Python backend and the JS frontend
I recommend installing everything in a virtual environment.
When using conda, even node/npm itself can be installed within the environment, thus not conflicting with any other installations.
### Python:
The Python requirements for this tool are listed in the ``requirements.txt`` file and are fairly basic.
Apart from those requirements listed there, additional requirements will be necessary depending on the middleware you want to use, currently supported are:
* TCP (already included in the standard library)
* ZMQ (requires zmq/pyzmq)
* RSB (requires rsb-python, **only Python2 support**)
* Ipaaca (requires the python ipaaca module, which requires rsb)
You can install all core requirements, e.g. using ``pip install -r requirements.txt``.
### Javascript:
When developing new components, or making other changes to the frontend, a ``node`` environment is required.
Node can be installed globally, or into a conda environment from ``conda-forge``:
``conda install -c conda-forge nodejs``
Once node and ``npm`` is installed, you can install the required JS dependencies, which are specified in the
``package.json`` located under ``webviz/static/``.
In case ``npm install`` fails when executed in that folder, delete the ``package-lock.json`` and the ``node_modules`` folder and try installing the dependencies manually one by one, some versions of npm seem to fail when recursively checking dependencies.
## Available components
### LinePlot
TODO
### BarPlot
TODO
### Slider
TODO
## Creating new components
While the collection of available tools should continue to grow, not all use-cases will always be covered. Maybe you have special needs for specific plots, or you want to create an interaction rich control panel. WebViz is trying to be very extendable by allowing basically any React component to be added to the rest of the tool.
I highly recommend running the development server of webpack using ``npm start`` within the ``static`` folder, as it will automatically detect changes made to your JS code and refresh the webpage. Don't worry that this development server will open its own website with a different port, the site will still communicate with the Python backend just fine.
### 1. Create your desired component
The first thing you should do is create a new react component for your use-case. This component may obviously make use of any number of other components or even set-up its own communication with other processes. In order to work well with other modules, it is however best, if the component is only dependent on props and its internal state. THis means the component should expect any data it requires for rendering to be available as a property.
As a very simple example, have a look at the ``checkedText`` component.
### 2. Create module configuration and connect to store
In order to allow the component to work with the rest of the tool and get data via backend, it needs to be connected to the Redux store and it needs to specify the kind of configuration it allows.
In order to connect it to the store, you will need to write a ``mapStateToProps`` function (a simple example can be seen in ``connectedCheckedText.js``). If your component does not only want to read parts of the store state, but also
write data there itself, you will further need to write a ``mapDispatchToProps`` function (an example for this can be seen in ``slider.js`` and more information regarding in [Writing custom data to the store](#writing-custom-data-to-the-store)). Afterwards you just need to use Redux's ``connect`` function to create the higher order component, as shown in ``slider.js`` or ``connectedCheckedText.js``.
In order to be able to configure your module, WebViz needs to know what kinds of options are possible. WebViz will automatically generate suitable configuration dialogs both for creating new modules and modifying already active modules. It does this by reading an ``Information-object`` which specifies all available options for your component.
For information, of how to structure this object refer to [this section](#the-information-object).
### 3. Make your module known to the dashboard
The final step you need to take, is register your new (connected) component to the ``moduleStore``. Ideally, we would like to automate this step, but currently it is still necessary to register them manually. The ``modules`` object needs to be extended by your new component. Furthermore, you will need to update the two mappings ``mapIDToConfig`` and ``mapTypeToComponent``.
Voilà, your new component should now appear under the modules tab and should be addable with your desired configurations.
## Technicalities
### The information object
Any component which should be useable as a module needs to have an associated information object, which will currently need to be registered manually to the moduleStore. As this is currently still very much in flux, it is best to look at examples, e.g. in ``slider.js`` or ``connectedCheckedText.js`` how it currently looks.
### Redux store structure
TODO
### Writing custom data to the store
TODO
\ No newline at end of file
DEBUG = True
HOST = "127.0.0.1"
PORT = 5000
\ No newline at end of file
File added
flask
flask-socketio
eventlet
\ No newline at end of file
from setuptools import setup, find_packages
setup(
name='WebViz',
version='0.1',
long_description=__doc__,
packages=find_packages(),
include_package_data=True,
zip_safe=False,
install_requires=['Flask', "Flask-socketio", "eventlet"],
extras_require={
"ipaaca": ["ipaaca"],
"zmq": ["zmq"],
"rsb": ["rsb-python"],
}
)
\ No newline at end of file
# -*- coding: utf-8 -*-
"""
Created on Sat Jun 24 00:45:04 2017
@author: Jan
"""
import random
import time
import socket
import json
from builtins import bytes
# import ipaaca
# ipaacaOutbuffer = ipaaca.OutputBuffer("IpaacaTest")
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
sock.sendall(bytes(message + "\n", "utf-8"))
sock.close()
try:
while True:
data = {"y": random.random(), # For Line data
"dist": list(enumerate([random.random() for i in range(5)])),
"dist2": {c: random.random() for c in "abcde"},
"y2": [random.random() for i in range(20)],
"x": ["a","b","c","d"],
"d": [random.random() for i in range(4)],
"checked": {m: True if random.random()>0.5 else False for m in ["Expl1", "Expl2", "Expl3"]}
}
idx = random.randint(0,4)
el = data["dist"][idx]
data["dist"][idx] = (el[0], el[1], "red")
try:
port = 9080
client("localhost", port, json.dumps(data))
except:
pass
# msg = ipaaca.Message("test")
# msg.payload = {k: str(v) for k,v in data.items()}
# msg.payload["x"] = ["a","b","c","d"]
# msg.payload["d"] = [random.random() for i in range(4)]
# ipaacaOutbuffer.add(msg)
time.sleep(0.15)
except KeyboardInterrupt:
pass
# del informer
# del ipaacaOutbuffer
#import sys
#sys.path.append("/homes/jpoeppel/repo/ipaaca/ipaacalib/python/build")
try:
import eventlet
eventlet.monkey_patch()
except ImportError:
pass
from webviz import create_app, socketio
app = create_app(debug=True)
if __name__ == "__main__":
socketio.run(app, host=app.config["HOST"], port=app.config["PORT"], debug= True, use_reloader=False)
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment