Processing a scene

The scene processing part of the toolbox defines methodes to transform an image into a place code in the sense of Basten and Mallot (2010).

The scene is either

  • a 4d numpy array, used when the image is a equirectangular projection, i.e. a panoramic image.
  • a 3d numpy array, used when the viewing direction can not be mapped/projected on a regular image (e.g. insect eye).

We thus define the following for a scene:

image based scene (IBS)
A classical image. Each pixel is viewed in a direction (elevation,azimuth) in a regular manner. In that case the scene is a 4d numpy array [elevation-index,azimuth-index,channel-index,1].
Omatidium based scene (OBS)
In an ommatidia based scene, the viewing direction do not need to be regularally spaced. In that case the scene is a 3d numpy array [ommatidia-index, channel-index,1].

By extension a place-code is either image based or ommatidium based. The number of dimension of an ib-place-code is always 4, and of an ob-place-code always 3.

image based place-code (IBPC)
A place code derived from IBS. Each pixel is viewed in a direction (elevation,azimuth) in a regular manner. In that case the scene is a 4d numpy array [elevation-index,azimuth-index,channel-index,component-index].
Omatidium based place-code (OBPC)
A place code derived from OBS, the viewing direction do not need to be regularally spaced. In that case the scene is a 3d numpy array [ommatidia-index, channel-index,component-index].

Abusing the terminology of a place-code, a scene can be a place-code. Therefore ibs and obs have 4 and 3 dimension, respectively.

Place code

Scene

Return a scene at a position orientation or given rowid in a given database.

Parameters:
  • database – a DataBaseLoad class :param posorient: a pandas Series with index: [‘x’,’y’,’z’,’alpha_0,’alpha_1,’alpha_2’] (default None, i.e. not used)
  • rowid – a row identification integer for directly reading in the database (default None, i.e. not used).
Returns:

a scene [elevation, azimuth, channel, 1] or [ommatidia,channel,1].

Return type:

np.ndarray

rowid = 12
my_scene = processing.pcode.scene(mydb, rowid=rowid)

(Source code, png, hires.png, pdf)

_images/scene.png

Skyline

Return the average along the elevation of a scene :param scene: the scenery at a given location (a 4d numpy array) :returns: the skyline [1,azimuth,channel,1] :rtype: np.ndarray

rowid = 12
my_scene = processing.scene(mydb, rowid=rowid)
my_skyline = processing.skyline(my_scene)

(Source code, png, hires.png, pdf)

_images/skyline.png

Michelson-contrast

Return the michelson constrast

\[\frac{I_\text{max}-I_\text{min}}{I_\text{max}+I_\text{min}}\]

with \(I_\text{max}\) and \(I_\text{min}\) representing the highest and lowest luminance in an image region around each pixel.

Parameters:
  • scene – an image based scene
  • size – the size of the region to calculate the maximum and minimum of the local image intensity
Returns:

the michelson-contrast

Return type:

np.ndarray

rowid = 12
my_scene = processing.scene(mydb, rowid=rowid)
my_contrast = processing.michelson_contrast(my_scene)

(Source code, png, hires.png, pdf)

_images/michelson_contrast.png

Contrast-weighted-nearness

Return the michelson contrast wheighted nearness

Parameters:
  • scene – an image based scene
  • contrast_size – the size of the region to calculate the maximum and minimum of the local image intensity in the michelson-contrast.
  • distance_channel – the index of the distance-channel.
rowid = 12
my_scene = processing.pcode.scene(mydb, rowid=rowid)
my_contrast = processing.pcode.contrast_weighted_nearness(my_scene)

(Source code, png, hires.png, pdf)

_images/contrast_weighted_nearness.png

Place-code vectors

Place code vectors

Parameters:
  • place_code – the place code at a given location (e.g. an ibs scene)
  • viewing_directions – viewing direction of each pixel
Returns:

the place code vectors in cartesian coordinates

Return type:

(np.ndarray)

rowid = 12
my_scene = processing.pcode.scene(mydb, rowid=rowid)
my_pcv = processing.pcode.pcv(my_scene, mydb.viewing_directions)

(Source code)

Average place-code vector

Calculate the average scene vector

Parameters:
  • place_code – the place code at a given location (e.g. an ibs scene)
  • viewing_directions – viewing direction of each pixel
Returns:

the average place-code vector

Return type:

(np.ndarray)

rowid = 12
my_scene = processing.pcode.scene(mydb, rowid=rowid)
my_apcv = processing.pcode.apcv(my_scene, mydb.viewing_directions)

(Source code, png, hires.png, pdf)

_images/apcv.png

Motion code

Optic flow

optic flow