IQR Classes

Here we list and briefly describe the high level algorithm interfaces which SMQTK-IQR provides. Some implementations will require additional dependencies that cannot be packaged with SMQTK-IQR.

IqrController

class smqtk_iqr.iqr.iqr_controller.IqrController(expire_enabled: bool = False, expire_check: float = 30, expire_callback: Optional[Callable] = None)

Main controlling object for one or more IQR Sessions.

In order to interface with a web server, methods defined here are thread-safe.

This class may be used with the with statement. This will enable the instance’s primary lock, preventing any other action from being performed on the instance while inside the with statement. The lock is reentrant, so nested with-statements will not dead-lock.

_handle_session_expiration() → None

Run on a separate thread periodic checks and removals of sessions that have expired.

add_session(iqr_session: IqrSession, timeout: float = 0) → Hashable

Initialize a new IQR Session, returning the uuid of that session

This controller indexes the given session by its UUID.

Parameters:

• iqr_session – The IqrSession instance to add

• timeout – The optional timeout, in seconds.

Returns:

UUID of new IQR Session

get_session(session_uuid: Hashable) → IqrSession

Return the session instance for the given UUID

Raises:

KeyError – The given UUID doesn’t exist in this controller.

Parameters:

session_uuid – UUID if the session to get

Returns:

IqrSession instance for the given UUID

has_session_uuid(session_uuid: Hashable) → bool

Check if this controller contains a session referenced by the given ID.

Performance using this function is faster compared to getting all UUIDs and performing a linear search (because hash tables).

This does NOT update session last access in regards to session expiration.

Parameters:

session_uuid – Possible UUID of a session

Returns:

True of the given UUID references a session in this controller and false if not.

remove_session(session_uuid: Hashable) → None

Remove an IQR Session by session UUID.

Raises:

KeyError – The given UUID doesn’t exist in this controller.

Parameters:

session_uuid – Session UUID

session_uuids() → Tuple

Return a tuple of all currently registered IqrSessions.

This does NOT update session last access in regards to session expiration.

Returns:

a tuple of all currently registered IqrSessions.

start_expiration_monitor() → None

Initialize unique thread to check for session expiration if the feature is enabled. This does nothing if the feature is not enabled.

We stop the previous thread if one was started.

stop_expiration_monitor() → None

Stop the session expiration monitoring thread if one has been started. Otherwise this method does nothing.

IqrSession

class smqtk_iqr.iqr.iqr_session.IqrSession(rank_relevancy_with_feedback: ~smqtk_relevancy.interfaces.rank_relevancy.RankRelevancyWithFeedback, pos_seed_neighbors: int = 500, session_uid: ~typing.Optional[~typing.Union[str, ~uuid.UUID]] = None, distance_metric: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], ~numpy.ndarray] = <function euclidean_distance>, autoneg_select_ratio: int = 1)

Encapsulation of IQR Session related data structures with a centralized lock for multi-thread access.

This object is compatible with the python with-statement, so when elements are to be used or modified, it should be within a with-block so race conditions do not occur across threads/sub-processes.

_ordered_pos: Optional[Sequence[Tuple[DescriptorElement, float]]]

Positively adjudicated descriptors in order of relevancy score.

adjudicate(new_positives: Iterable[DescriptorElement] = (), new_negatives: Iterable[DescriptorElement] = (), un_positives: Iterable[DescriptorElement] = (), un_negatives: Iterable[DescriptorElement] = ()) → None

Update current state of working set positive and negative adjudications based on descriptor UUIDs.

If the same descriptor element is listed in both new positives and negatives, they cancel each other out, causing that descriptor to not be included in the adjudication.

The given iterables must be re-traversable. Otherwise the given descriptors will not be properly registered.

Parameters:

• new_positives – Descriptors of elements in our working set to now be considered to be positively relevant. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

• new_negatives – Descriptors of elements in our working set to now be considered to be negatively relevant. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

• un_positives – Descriptors of elements in our working set to now be considered not positive any more. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

• un_negatives – Descriptors of elements in our working set to now be considered not negative any more. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

external_descriptors(positive: Iterable[DescriptorElement] = (), negative: Iterable[DescriptorElement] = ()) → None

Add positive/negative descriptors from external data.

These descriptors may not be a part of our working set.

TODO: Add ability to “remove” positive/negative external descriptors.

See adjudicate method “un_…” parameters.

Parameters:

• positive – Iterable of descriptors from external sources to consider positive examples.

• negative – Iterable of descriptors from external sources to consider negative examples. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

feedback_results() → List[DescriptorElement]

Return a list of all working-set descriptor elements that would benefit from further refinement. The list is in order of most to least useful.

If refinement has not yet occurred since session creation or the last reset, an empty tuple is returned.

Raises:

RuntimeError – If the end of the function is reached this means the feedback results have gotten into an invalid state.

get_negative_adjudication_relevancy() → List[Tuple[DescriptorElement, float]]

Return a list of the negatively adjudicated descriptors as tuples of (element, score) in order of descending relevancy score.

This does not include external negative adjudications, only negatively adjudicated descriptors in the working set.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

Cache is invalidated when: - A refinement occurs. - Negative adjudications change.

get_positive_adjudication_relevancy() → List[Tuple[DescriptorElement, float]]

Return a list of the positively adjudicated descriptors as tuples of (element, score) in order of descending relevancy score.

This does not include external positive adjudications, only positively adjudicated descriptors in the working set.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

Cache is invalidated when: - A refinement occurs. - Positive adjudications change.

get_state_bytes() → bytes

Get a byte representation of the current descriptor and adjudication state of this session.

This does not encode current results or the relevancy index’s state, but these can be reproduced with this state.

Returns:

State representation bytes

get_unadjudicated_relevancy() → List[Tuple[DescriptorElement, float]]

Return a list of the non-adjudicated descriptor elements as tuples of (element, score) in order of descending relevancy score.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

ordered_results() → List[Tuple[DescriptorElement, float]]

Return a tuple of all working-set descriptor elements as tuples of (element, score) in order of descending relevancy score.

If refinement has not yet occurred since session creation or the last reset, an empty tuple is returned.

refine() → None

Refine current model results based on current adjudication state

Raises:

• RuntimeError – No working set has been initialized. update_working_set() should have been called after adjudicating some positive examples.

• RuntimeError – There are no adjudications to run on. We must have at least one positive adjudication.

reset() → None

Reset the IQR Search state

No positive adjudications, reload original feature data

set_state_bytes(b: bytes, descriptor_factory: DescriptorElementFactory) → None

Set this session’s state to the given byte representation, resetting this session in the process.

Bytes given must have been retrieved via a previous call to get_state_bytes otherwise this method will fail.

Since this state may be completely different from the current state, this session is reset before applying the new state. Thus, any current ranking results are thrown away.

Parameters:

• b – Bytes to set this session’s state to.

• descriptor_factory – Descriptor element factory to use when generating descriptor elements from extracted data.

Raises:

ValueError – The input bytes could not be loaded due to incompatibility.

update_working_set(nn_index: NearestNeighborsIndex) → None

Initialize or update our current working set using the given NearestNeighborsIndex instance given our current positively labeled descriptor elements.

We only query from the index for new positive elements since the last update or reset.

Parameters:

nn_index – NearestNeighborsIndex to query from.

Raises:

RuntimeError – There are no positive example descriptors in this session to use as a basis for querying.