IQR Classes

Here we list and briefly describe the high level algorithm interfaces which SMQTK-IQR provides. Some implementations will require additional dependencies that cannot be packaged with SMQTK-IQR.

IqrController

class smqtk_iqr.iqr.iqr_controller.IqrController(expire_enabled: bool = False, expire_check: float = 30, expire_callback: Optional[Callable] = None)[source]

Main controlling object for one or more IQR Sessions.

In order to interface with a web server, methods defined here are thread-safe.

This class may be used with the with statement. This will enable the instance’s primary lock, preventing any other action from being performed on the instance while inside the with statement. The lock is reentrant, so nested with-statements will not dead-lock.

_handle_session_expiration() → None[source]

Run on a separate thread periodic checks and removals of sessions that have expired.

add_session(iqr_session: smqtk_iqr.iqr.iqr_session.IqrSession, timeout: float = 0) → collections.abc.Hashable[source]

Initialize a new IQR Session, returning the uuid of that session

This controller indexes the given session by its UUID.

Parameters

  • iqr_session – The IqrSession instance to add

  • timeout – The optional timeout, in seconds.

Returns

UUID of new IQR Session

get_session(session_uuid: collections.abc.Hashable) → smqtk_iqr.iqr.iqr_session.IqrSession[source]

Return the session instance for the given UUID

Raises

KeyError – The given UUID doesn’t exist in this controller.

Parameters

session_uuid – UUID if the session to get

Returns

IqrSession instance for the given UUID

has_session_uuid(session_uuid: collections.abc.Hashable) → bool[source]

Check if this controller contains a session referenced by the given ID.

Performance using this function is faster compared to getting all UUIDs and performing a linear search (because hash tables).

This does NOT update session last access in regards to session expiration.

Parameters

session_uuid – Possible UUID of a session

Returns

True of the given UUID references a session in this controller and false if not.

remove_session(session_uuid: collections.abc.Hashable) → None[source]

Remove an IQR Session by session UUID.

Raises

KeyError – The given UUID doesn’t exist in this controller.

Parameters

session_uuid – Session UUID

session_uuids() → Tuple[source]

Return a tuple of all currently registered IqrSessions.

This does NOT update session last access in regards to session expiration.

Returns

a tuple of all currently registered IqrSessions.

start_expiration_monitor() → None[source]

Initialize unique thread to check for session expiration if the feature is enabled. This does nothing if the feature is not enabled.

We stop the previous thread if one was started.

stop_expiration_monitor() → None[source]

Stop the session expiration monitoring thread if one has been started. Otherwise this method does nothing.

IqrSession

class smqtk_iqr.iqr.iqr_session.IqrSession(rank_relevancy_with_feedback: smqtk_relevancy.interfaces.rank_relevancy.RankRelevancyWithFeedback, pos_seed_neighbors: int = 500, session_uid: Optional[Union[str, uuid.UUID]] = None)[source]

Encapsulation of IQR Session related data structures with a centralized lock for multi-thread access.

This object is compatible with the python with-statement, so when elements are to be used or modified, it should be within a with-block so race conditions do not occur across threads/sub-processes.

_ordered_pos: Optional[Sequence[Tuple[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement, float]]]

Positively adjudicated descriptors in order of relevancy score.

adjudicate(new_positives: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = (), new_negatives: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = (), un_positives: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = (), un_negatives: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = ()) → None[source]

Update current state of working set positive and negative adjudications based on descriptor UUIDs.

If the same descriptor element is listed in both new positives and negatives, they cancel each other out, causing that descriptor to not be included in the adjudication.

The given iterables must be re-traversable. Otherwise the given descriptors will not be properly registered.

Parameters

  • new_positives – Descriptors of elements in our working set to now be considered to be positively relevant. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

  • new_negatives – Descriptors of elements in our working set to now be considered to be negatively relevant. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

  • un_positives – Descriptors of elements in our working set to now be considered not positive any more. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

  • un_negatives – Descriptors of elements in our working set to now be considered not negative any more. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

external_descriptors(positive: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = (), negative: Iterable[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement] = ()) → None[source]

Add positive/negative descriptors from external data.

These descriptors may not be a part of our working set.

TODO: Add ability to “remove” positive/negative external descriptors.

See adjudicate method “un_…” parameters.

Parameters

  • positive – Iterable of descriptors from external sources to consider positive examples.

  • negative – Iterable of descriptors from external sources to consider negative examples. collections.abc.Iterable[smqtk_descriptors.DescriptorElement]

feedback_results() → List[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement][source]

Return a list of all working-set descriptor elements that would benefit from further refinement. The list is in order of most to least useful.

If refinement has not yet occurred since session creation or the last reset, an empty tuple is returned.

Raises

RuntimeError – If the end of the function is reached this means the feedback results have gotten into an invalid state.

get_negative_adjudication_relevancy() → List[Tuple[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement, float]][source]

Return a list of the negatively adjudicated descriptors as tuples of (element, score) in order of descending relevancy score.

This does not include external negative adjudications, only negatively adjudicated descriptors in the working set.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

Cache is invalidated when: - A refinement occurs. - Negative adjudications change.

get_positive_adjudication_relevancy() → List[Tuple[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement, float]][source]

Return a list of the positively adjudicated descriptors as tuples of (element, score) in order of descending relevancy score.

This does not include external positive adjudications, only positively adjudicated descriptors in the working set.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

Cache is invalidated when: - A refinement occurs. - Positive adjudications change.

get_state_bytes() → bytes[source]

Get a byte representation of the current descriptor and adjudication state of this session.

This does not encode current results or the relevancy index’s state, but these can be reproduced with this state.

Returns

State representation bytes

get_unadjudicated_relevancy() → List[Tuple[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement, float]][source]

Return a list of the non-adjudicated descriptor elements as tuples of (element, score) in order of descending relevancy score.

If refinement has not yet occurred since session creation or the last reset, an empty list is returned.

ordered_results() → List[Tuple[smqtk_descriptors.interfaces.descriptor_element.DescriptorElement, float]][source]

Return a tuple of all working-set descriptor elements as tuples of (element, score) in order of descending relevancy score.

If refinement has not yet occurred since session creation or the last reset, an empty tuple is returned.

refine() → None[source]

Refine current model results based on current adjudication state

Raises

  • RuntimeError – No working set has been initialized. update_working_set() should have been called after adjudicating some positive examples.

  • RuntimeError – There are no adjudications to run on. We must have at least one positive adjudication.

reset() → None[source]

Reset the IQR Search state

No positive adjudications, reload original feature data

set_state_bytes(b: bytes, descriptor_factory: smqtk_descriptors.descriptor_element_factory.DescriptorElementFactory) → None[source]

Set this session’s state to the given byte representation, resetting this session in the process.

Bytes given must have been retrieved via a previous call to get_state_bytes otherwise this method will fail.

Since this state may be completely different from the current state, this session is reset before applying the new state. Thus, any current ranking results are thrown away.

Parameters

  • b – Bytes to set this session’s state to.

  • descriptor_factory – Descriptor element factory to use when generating descriptor elements from extracted data.

Raises

ValueError – The input bytes could not be loaded due to incompatibility.

update_working_set(nn_index: smqtk_indexing.interfaces.nearest_neighbor_index.NearestNeighborsIndex) → None[source]

Initialize or update our current working set using the given NearestNeighborsIndex instance given our current positively labeled descriptor elements.

We only query from the index for new positive elements since the last update or reset.

Parameters

nn_index – NearestNeighborsIndex to query from.

Raises

RuntimeError – There are no positive example descriptors in this session to use as a basis for querying.