Decision Preferences

As described in Decision Process (DP), the Rank phase provides knowledge to select from amongst multiple proposed actions. In the code below, three “Choice” operators (a, b, and c) are proposed in a single-decision DP - the Cogent is run multiple items to illustrate the result of utilizing different preference-knowledge approaches:

  1"""
  2Evaluating choices
  3"""
  4
  5from typing import Any, cast
  6
  7from cognition import (
  8    Action,
  9    Cogent,
 10    DecisionProcess,
 11    IOContainer,
 12    NamedObject,
 13    Operator,
 14    Rank,
 15    operator_sorting_key,
 16    sorting_evaluator,
 17    uniform_evaluator,
 18)
 19
 20# ===
 21
 22PARAM_CHOICE = "result"
 23
 24
 25class Choice(Operator[str]):
 26    """one-step decision"""
 27
 28    def __init__(self, result: str) -> None:
 29        super().__init__(
 30            type(self).__name__, **{PARAM_CHOICE: result, "terminal": True}
 31        )
 32
 33    def can_perform(self, _state: str, _io: IOContainer) -> bool:
 34        return True
 35
 36    def perform(self, _state: str, _io: IOContainer) -> str:
 37        return self.params[PARAM_CHOICE]
 38
 39    # START op_sorting_defn
 40    def __eq__(self, other) -> bool:
 41        if not isinstance(other, Choice):
 42            return NotImplemented
 43
 44        return self.params[PARAM_CHOICE] == other.params[PARAM_CHOICE]
 45
 46    def __lt__(self, other) -> bool:
 47        if not isinstance(other, Choice):
 48            return NotImplemented
 49
 50        return self.params[PARAM_CHOICE] < other.params[PARAM_CHOICE]
 51
 52    # END op_sorting_defn
 53
 54
 55# ===
 56
 57
 58def all_same(dp: DecisionProcess) -> None:
 59    """all operators get `MEDIUM` ranking"""
 60
 61    # START all_same
 62    dp.add_action_evaluator(uniform_evaluator(Rank.MEDIUM))
 63    # END all_same
 64
 65
 66def op_classes(dp: DecisionProcess) -> None:
 67    """mix uniform via predicate: a=LOW, other=HIGH"""
 68
 69    # START op_classes
 70    def is_a(a: Any) -> bool:
 71        return cast(NamedObject, a).params[PARAM_CHOICE] == "a"
 72
 73    def not_a(a: Any) -> bool:
 74        return not is_a(a)
 75
 76    dp.add_action_evaluator(uniform_evaluator(Rank.LOW, p=is_a))
 77    dp.add_action_evaluator(uniform_evaluator(Rank.HIGH, p=not_a))
 78    # END op_classes
 79
 80
 81def op_sorting_abs(dp: DecisionProcess) -> None:
 82    """explicit association of rank to values"""
 83
 84    # START op_sorting_abs
 85    def _key(a: Action[str], _state: str, _io: IOContainer) -> int:
 86        return {
 87            "a": 100,
 88            "b": 50,
 89            "c": 1,
 90            "d": 0,  # ok that it doesn't exist
 91        }[cast(NamedObject, a).params[PARAM_CHOICE]]
 92
 93    dp.add_action_evaluator(sorting_evaluator(_key))
 94    # END op_sorting_abs
 95
 96
 97def op_sorting_rel(dp: DecisionProcess) -> None:
 98    """rank via relative object-based comparison"""
 99
100    # START op_sorting_rel
101    dp.add_action_evaluator(sorting_evaluator(operator_sorting_key()))
102    # END op_sorting_rel
103
104
105# ===
106
107cogent = Cogent(DecisionProcess(lambda: "_start_"))
108
109cogent.dp.add_operator(Choice("a"))
110cogent.dp.add_operator(Choice("b"))
111cogent.dp.add_operator(Choice("c"))
112
113# TODO: choose an evaluation function above!
114# (None = CRASH due multiple options, lack of ranking)
115# all_same(cogent.dp)  # all same rank (so random)
116# op_classes(cogent.dp)  # categories of rank
117# op_sorting_abs(cogent.dp)  # absolute
118op_sorting_rel(cogent.dp)  # relative
119
120# ===
121
122print(f"{cogent.dp.state} -> ")
123for i in range(5):
124    cogent()
125    print(f"{i+1}) {cogent.dp.state=}")

Ranking Semantics

  • Any comparable value can be utilized for rank values

    • Smaller values are considered more desirable

    • For convenience, the Rank enumeration is included for simple cases: LOW/MEDIUM/HIGH (where LOW has the largest value).

  • Multiple evaluators can be utilized - the decision is made based upon the union of ActionRank objects produced.

Uniform -> Random

    dp.add_action_evaluator(uniform_evaluator(Rank.MEDIUM))

The uniform_evaluator function produces an evaluator that provides the supplied rank value to ALL proposed actions.

Thus, using this strategy produces a mix of a/b/c results.

Uniform + Predicate -> Categories

    def is_a(a: Any) -> bool:
        return cast(NamedObject, a).params[PARAM_CHOICE] == "a"

    def not_a(a: Any) -> bool:
        return not is_a(a)

    dp.add_action_evaluator(uniform_evaluator(Rank.LOW, p=is_a))
    dp.add_action_evaluator(uniform_evaluator(Rank.HIGH, p=not_a))

The uniform_evaluator function optionally accepts a predicate to filter those actions to which an instance supplies ranking information.

In this case there are two categories: “a” (LOW rank) and all others (HIGH rank), and thus this strategy produces a mix of b/c results.

Sorting: Absolute Values -> Relative Ranks

    def _key(a: Action[str], _state: str, _io: IOContainer) -> int:
        return {
            "a": 100,
            "b": 50,
            "c": 1,
            "d": 0,  # ok that it doesn't exist
        }[cast(NamedObject, a).params[PARAM_CHOICE]]

    dp.add_action_evaluator(sorting_evaluator(_key))

The sorting_evaluator function produces an evaluator that sorts the proposed actions (using a required sorting_key function) and then automatically produces appropriate ranks based upon the relative ordering.

In this case, the sorting key is based upon a hard-coded dictionary associating action -> value; this strategy thus always produces “c” as the final result (since it will be deterministically sorted first amongst the options).

Note

Given this structure, cogents can adaptively modify decision-selection knowledge based upon feedback (e.g., via TD Learning).

Sorting: Object Comparison -> Relative Ranks

A useful companion to the sorting_evaluator function is operator_sorting_key:

    dp.add_action_evaluator(sorting_evaluator(operator_sorting_key()))

This combination will expect that objects can be pairwise compared (via __eq__ and __lt__):

    def __eq__(self, other) -> bool:
        if not isinstance(other, Choice):
            return NotImplemented

        return self.params[PARAM_CHOICE] == other.params[PARAM_CHOICE]

    def __lt__(self, other) -> bool:
        if not isinstance(other, Choice):
            return NotImplemented

        return self.params[PARAM_CHOICE] < other.params[PARAM_CHOICE]

In this case, the objects will sort lexigraphically based upon action result, thereby always producing “a” as the final decision.