diff --git a/doc/ant_colony_mining.md b/doc/ant_colony_mining.md
new file mode 100644
index 000000000..b1c1336ef
--- /dev/null
+++ b/doc/ant_colony_mining.md
@@ -0,0 +1,375 @@
+# Ant Colony Mining
+
+This document has two parts:
+
+- **Part 1 - Overview**: what ant-colony mining is.
+- **Part 2 - Miner / pool integration guide**: the exact on-the-wire contract.
+
+> Config values (threshold, freshness window, child cap) are per-epoch and can change between epochs.
+> Always read the live values from the epoch-context query (Part 2, section 2.7a). Numbers quoted in
+> this document are current defaults, not constants to hard-code.
+
+> **Reading this doc.** Part 1 explains the split between the ant-colony *structure* and the mining
+> *algorithm*. The parts specific to today's algorithm - the score's range and direction, the nonce
+> knobs and their ranges, the mutation walk, and the numeric constants - are labelled **bpp9000** below;
+> do not treat them as fixed ant-colony rules.
+
+---
+
+## Part 1 - Overview
+
+**Ant colony is not a mining algorithm - it is a search *structure*.** It does not decide what a good
+solution looks like or how to score one - a **mining algorithm** does. `nonce[0]` selects which
+algorithm runs; today the only one is **bpp9000**, which the ant colony runs by default. A future
+algorithm could be added the same way, in the same structure. So "ant colony" and "bpp9000" are two
+separate things: the structure, and the algorithm currently running in it.
+
+| The ant-colony **structure** provides (any algorithm) | The **algorithm** provides (bpp9000 today) |
+|---|---|
+| a per-identity tree of solutions, grown from parents | what a solution *is* |
+| the accept rules, the deposit, and the ranking | how to derive a child solution from a parent |
+| the request / response queries | how to score a solution |
+
+With that split in mind: mining is a search for a **solution** that does well on a fixed task, and the
+algorithm defines what a solution is and how it scores. Under **bpp9000** a solution is a neural network
+(an "ANN"), scored by an **error count** over the task's data windows - range `[0, 8088]`, **lower is
+better** (a flawless network makes zero mistakes). The rest of this overview uses bpp9000's terms, but
+the tree structure around them is identical for any algorithm.
+
+Under bpp9000 the network's **wiring** (which neuron reads which) is **global** - the same graph for
+everyone, from the epoch's task file - and a miner searches over each neuron's **lookup table (LUT)**,
+the ternary function it computes.
+
+Standalone mining searches alone: every attempt starts from scratch. Ant-colony mining searches
+**together, as a tree**:
+
+- Every **mining identity** (a computor or candidate public key) owns its **own tree** - the colony is
+ a per-identity forest. A pool's workers extend the tree of the computor they mine for.
+- Each tree starts from a **virtual root**: a starting solution derived from that identity's public key
+ and the epoch's spectrum digest. It is fixed for the epoch, identical every time you derive it, and
+ is never stored or submitted.
+- To mine, you pick a **parent** (the root, or any node already in your tree), **inherit** it, vary it
+ under your nonce, and score the result.
+- If the result **strictly beats the parent** and clears the epoch **threshold**, you **submit** it. On
+ acceptance it becomes a new tree node that you - or anyone - can extend further.
+
+So the colony converges toward better solutions: each accepted node beats its parent, and the next
+miner starts from there instead of from scratch. The goal of the epoch is the single best solution
+found anywhere in the forest.
+
+```
+ virtual root (per identity, not stored)
+ |
+ +----+----+
+ | |
+ node A node B each node beats its parent
+ | | and clears the threshold
+ node C node D
+ |
+ node E <-- best in this tree so far
+```
+
+Concretely, error gates every attachment: it only falls down a branch (a child must beat its parent),
+and a *start* - a depth-1 child of the root - must clear the threshold.
+
+```
+ error = error count, lower is better threshold = 3838
+
+ root ~4044 raw a fresh root sits above 3838; a start must mutate below it
+ |
+ +-- A 3790 <= threshold ACCEPT (depth-1 start)
+ | |
+ | +-- B 3540 < 3790, beats A ACCEPT
+ | | |
+ | | +-- D 3120 < 3540, beats B ACCEPT
+ | | +-- E 3560 not < 3540 REJECT (must beat parent)
+ | |
+ | +-- C 3700 < 3790, beats A ACCEPT
+ |
+ +-- X 3900 > threshold REJECT (over threshold)
+
+ Error only falls as you go deeper. The epoch winner is the single lowest-error node
+ found in any identity's forest.
+```
+
+At epoch end the node ranks every identity by its **single best** score and **harvests the top 676**
+(the number of computors).
+
+**Anti-spam deposit.** Each solution a computor publishes on-chain carries a **refundable
+1,000,000 QU deposit**, funded by the computor - not the worker. It is returned when the solution is
+accepted **and** its claimed score matches the node's recompute; otherwise it is kept. So a computor
+only publishes solutions it has already validated, and an honest, correct one costs nothing.
+
+---
+
+## Part 2 - Miner / pool integration guide
+
+**In short.** A miner works one identity's tree. It reads the epoch context, takes a **parent** (the
+identity's virtual root, or a node already in the tree), picks a canonical **nonce**, inherits the
+parent's network, and **mutates and scores** it - reproducing the node's score exactly. If the result
+**beats its parent** and **clears the threshold**, it hands the solution to the **computor**, which
+re-checks it and **publishes it on-chain**; every node then recomputes the score, folds it into
+consensus, attaches the node to the tree, and refunds the deposit. A miner's entire job is an **exact
+scorer** - the tree, gates, deposit, and queries are the wrapper around it.
+
+### 2.1 The mining loop
+
+1. **Epoch context** - `REQUEST_ANT_EPOCH_CONTEXT` (public). Read the threshold, freshness window,
+ epoch spectrum digest, and child cap for this epoch, and **verify your task file** against the
+ returned `topologyHash` / `dataHash` (section 2.7a) before doing any work.
+2. **Get a starting point** - derive your identity's virtual root, or fetch an existing node you want
+ to extend (`REQUEST_ANT_PARENT_ANN`).
+3. **Pick a parent** - the root, or any node in your own tree.
+4. **Search** - choose a nonce (section 2.2), inherit the parent LUT, run the mutation walk, score
+ (section 2.3).
+5. **Submit** - if it passes the local rules (section 2.4), send `AntSolutionBroadcastPayload` to the
+ computor you mine for (section 2.6, stage 1).
+6. **Publish + confirm** - the computor validates and scores it, then publishes it on-chain as an
+ `AntColonyMiningSolutionTransaction` (section 2.6, stage 2). Every node processes that transaction -
+ recompute, fold the digest, commit to the computor's tree, refund the deposit. Query the tree
+ (section 2.7b) to see accepted nodes and extend them.
+
+### 2.2 The nonce (32 bytes)
+
+| Byte(s) | Meaning | Valid range |
+|-------------|---------|-------------|
+| `nonce[0]` | algorithm selector (must select bpp9000) | - |
+| `nonce[1]` | `L` = LUT entries rewritten per mutation step | `[1, 10]` |
+| `nonce[2]` | `K` = number of **explore** steps | `[0, 100]` |
+| `nonce[3..31]` | the walk seed (the actual search space) | any |
+
+**Canonical-nonce rule.** The scorer **refuses** any non-canonical nonce - it returns no score, and the
+node rejects the submission with `RejectNonCanonicalNonce`. For an ant solution the rule is:
+
+```
+algo == bpp9000 && nonce[1] in [1, 10] && nonce[2] in [0, 100]
+```
+
+`nonce[0..2]` (the algo / `L` / `K` knobs) are **excluded from the RNG seed** - zeroed before hashing -
+so `L` and `K` can be chosen without changing the walk seed. This makes the score-relevant bytes equal
+to the identity/dedup bytes: there is no malleability room, and two nonces that differ only in these
+knobs are not two solutions.
+
+### 2.3 Scoring - bpp9000 (must be bit-exact)
+
+Throughout, `publicKey` is the **mining identity you are extending** - the computor you mine for, which
+becomes the transaction's `sourcePublicKey`. Derive the root and the mutation seed from **that** key,
+not your worker key, or the node's recompute will not match yours.
+
+**Root.** `deriveRootANN(publicKey, epochPool)`: `K12(publicKey)` seeds a per-neuron LUT from the
+epoch's random pool (the pool comes from the epoch-start spectrum digest). No mutation walk. Never
+stored. The same every time for the epoch.
+
+**Child.** `computeScoreFromParent(parentLUT, publicKey, nonce, anchorTickDigest)`:
+
+1. Inherit `parentLUT`.
+2. `mutationSeed = K12(publicKey || nonce[3..31] || anchorTickDigest)` (`nonce[0..2]` zeroed).
+3. Walk `numberOfMutations = 100` steps. Each step rewrites `L` LUT entries. For the first `K` steps
+ accept a worse-or-equal result (**explore**); after that accept only better-or-equal (**exploit**);
+ one-step rollback on reject. Keep and return the **best** score seen. The best is seeded with the
+ inherited network's own score, so a child that fails to improve on its parent is rejected (see
+ `RejectLeParent`).
+
+**Anchor digest.** `anchorTickDigest = K12(anchorTick || transactionDigest)`, where `transactionDigest`
+is `K12(TickData)` of the anchor tick's `TickData` (`REQUEST_TICK_DATA`). This binds a solution to a tick.
+
+Score is an error count in `[0, 8088]`; lower is better.
+
+### 2.4 Accept rules
+
+A submission is accepted (`Valid` or `ValidNotStored`) only if **all** of these hold. The node checks
+in this order; the first failure is the reject reason:
+
+| Check | Reject reason if it fails |
+|-------|---------------------------|
+| Parent record exists | `RejectParentNotRegistered` |
+| Parent is in the **same** identity's tree (the tx `sourcePublicKey`) | `RejectWrongTree` |
+| Nonce is canonical | `RejectNonCanonicalNonce` |
+| Anchor not in the future, published within `freshnessWindow` ticks of it | `RejectStale` / `RejectTickOutOfRange` |
+| Score `<=` epoch threshold | `RejectBelowThreshold` |
+| Score **strictly** below the parent's score | `RejectLeParent` |
+| Parent holds fewer than `maxChildrenPerParent` children (`0` = unbounded) | `RejectMaxChildrenPerParent` |
+| `(publicKey, parentRef, nonce)` not already committed this epoch | `RejectReplay` |
+| Store and miner index not full | `RejectDedupFull` / `RejectMinerIndexFull` |
+
+Separately, the **claimed score must equal the node's recompute** - if not, the solution may still be
+recorded but the **deposit is kept** and the miner is **not ranked**.
+
+**Starting a tree.** The root's record score is the worst possible value, so a first (depth-1) child
+passes the "beats parent" check trivially - the **threshold is the only score gate** for starting a
+tree. A random root scores far above the threshold, so a start still requires real mutation.
+
+**`ValidNotStored`.** Accepted, refunded, and ranked exactly like `Valid`, but the per-epoch store was
+full so the node was not persisted for others to extend. Ranking and refund are unaffected.
+
+### 2.5 The deposit
+
+Every on-chain `AntColonyMiningSolutionTransaction` carries a **1,000,000 QU** deposit
+(`SOLUTION_SECURITY_DEPOSIT`), funded by the **computor** that publishes it - not the miner (see 2.6).
+It is refunded **iff** the solution is accepted (`Valid` / `ValidNotStored`) **and** the claimed score
+equals the node's recompute; otherwise it is kept. So a computor risks its own deposit and therefore
+pre-validates each solution before publishing; a worker posts nothing on-chain. Keep the computor
+identity funded above the deposit.
+
+### 2.6 Submission: broadcast (off-chain), then transaction (on-chain)
+
+A solution travels in two stages - an off-chain hand-off to a computor, then the on-chain transaction
+that is the actual consensus record.
+
+**Stage 1 - P2P broadcast (`AntSolutionBroadcastPayload`, 48 bytes).** The miner hands its solution to
+the computor it mines for, inside the standard `BroadcastMessage` envelope (network type
+`BROADCAST_MESSAGE`), message type `MESSAGE_TYPE_ANT_SOLUTION` (`3`):
+
+```
+BroadcastMessage { // 96-byte envelope
+ m256i sourcePublicKey; // the worker key - pool off-chain accounting only, NOT the tree
+ m256i destinationPublicKey; // the COMPUTOR you mine for - THIS identity owns the tree and funds the deposit
+ m256i gammingNonce; // first gamming byte selects MESSAGE_TYPE_ANT_SOLUTION (3)
+}
+// then the payload:
+AntSolutionBroadcastPayload { // 48 bytes
+ unsigned int parentTick; // ABSOLUTE tick of the parent node (0 with the index below = root)
+ unsigned int parentSolutionIndexInTick;
+ unsigned int anchorTick; // ABSOLUTE tick number
+ unsigned int claimedScore;
+ m256i nonce; // the 32-byte nonce from 2.2
+}
+```
+
+The computor scores and validates on receipt (a non-canonical or already-seen solution is dropped for
+free). Nothing is on-chain yet.
+
+**Stage 2 - on-chain transaction (`AntColonyMiningSolutionTransaction`, `inputType` 12).** When the
+computor publishes, it emits a standard transaction into tick data under **its own** key and funds the
+deposit from **its own** balance - the computor pays, not the miner. This transaction is the consensus
+record: every node processes it in `processTick`, recomputes the score, folds it into
+`resourceTestingDigest`, commits the node to the computor's tree, and refunds or keeps the deposit.
+
+```
+AntColonyMiningSolutionTransaction : Transaction { // 80-byte header + 48-byte payload + 64-byte signature
+ // --- Transaction header ---
+ m256i sourcePublicKey; // the COMPUTOR (tree owner); signs the tx and funds the deposit
+ m256i destinationPublicKey; // zero (NULL_ID)
+ long long amount; // SOLUTION_SECURITY_DEPOSIT = 1,000,000 QU
+ unsigned int tick; // publish tick
+ unsigned short inputType; // ANT_COLONY_MINING_SOLUTION_INPUT_TYPE = 12
+ unsigned short inputSize; // 48
+ // --- payload (48 bytes) ---
+ unsigned int parentTick; // ABSOLUTE tick of the parent node
+ unsigned int parentSolutionIndexInTick;
+ unsigned int anchorTick; // ABSOLUTE
+ unsigned int claimedScore;
+ m256i nonce;
+ // --- 64-byte signature over header + payload ---
+}
+```
+
+`parentRef = (parentTick, parentSolutionIndexInTick) = (0, 0xFFFFFFFF)` means the **virtual root**
+(a depth-1 child).
+
+**Every tick in the protocol is absolute.** `parentTick`, `selfTick` and `anchorTick` are all real
+system tick numbers - the same values `getCurrentTick` or a tick-data query returns. A parent is named
+by the absolute tick it was committed in (plus its index within that tick), so the ref you copy from
+the identity tree is used verbatim - there is no epoch-relative offset to convert.
+
+**For a pool.** The tree belongs to the **computor** - `destinationPublicKey` of the broadcast, which
+becomes `sourcePublicKey` of the transaction. Workers hold no tree and post no on-chain deposit; they
+hand solutions to the pool's computor, which pre-validates them and risks its own deposit only on
+solutions it expects to be accepted and refunded. Fund the computor identity, not the workers.
+
+### 2.7 Read queries
+
+Three request/response pairs. **Identity tree** and **parent ANN** are **operator-signed**; **epoch
+context** is **public**.
+
+Operator signing: the request payload is followed by a **64-byte signature** over `K12(payload)`,
+verified against the node's configured **operator public key**. This lets a pool route and filter its
+own miners' reads while keeping untrusted parties from spamming the node. There is no monotonic nonce -
+replaying a read only costs a duplicate answer.
+
+```
+digest = K12(requestPayload)
+signature = sign(operatorSubseed, operatorPublicKey, digest) // 64 bytes, appended to the payload
+```
+
+**(a) Epoch context** - `REQUEST_ANT_EPOCH_CONTEXT` (76) / `RESPOND_ANT_EPOCH_CONTEXT` (77). **Public.**
+
+Request: empty. Response `RespondAntEpochContext` (120 bytes, packed):
+
+```
+m256i spectrumDigest; // epoch-start spectrum digest (seeds every root)
+m256i topologyHash; // canonical task topology-block hash (BPP9000_TOPOLOGY_HASH)
+m256i dataHash; // canonical task data-block hash (BPP9000_DATA_HASH)
+unsigned int threshold; // per-epoch accept bound
+unsigned int freshnessWindow; // publish within this many ticks of the anchor
+unsigned int solutionCount; // accepted solutions so far this epoch
+unsigned int freeAnnSlotsCount; // free slots in the live ANN pool
+unsigned int maxChildrenPerParent; // per-parent child cap; 0 = unbounded
+unsigned short epoch;
+unsigned short padding;
+```
+
+`topologyHash` / `dataHash` identify the exact task the node scores against. After loading your task
+file, recompute K12 over your own topology and data blocks and compare against these. If either
+differs you are holding the wrong task: **stop**. Mining a stale task wastes work, and the mismatched
+score forfeits the computor's deposit on every submission (the node scores with *its* task, so your
+claimed score never matches).
+
+**(b) Identity tree** - `REQUEST_ANT_IDENTITY_TREE` (72) / `RESPOND_ANT_IDENTITY_TREE` (73).
+**Operator-signed.** Paginated.
+
+Request `RequestAntIdentityTree` (40 bytes) + 64-byte signature:
+
+```
+m256i pubkey; // whose tree to report (usually your own)
+unsigned int fromIndex; // resume cursor, 0 on the first call
+unsigned int padding;
+```
+
+Response: `RespondAntIdentityTreeHeader` (12 bytes: `count`, `itemSize`, `nextIndex`) followed by
+`count` x `AntIdentityTreeNode`. Up to 64 nodes per response; page with `nextIndex` until it is `0`.
+
+```
+AntIdentityTreeNode { // 32 bytes
+ unsigned int selfTick; // ABSOLUTE; set these two as your parentRef to extend THIS node
+ unsigned int selfSolutionIndexInTick;
+ unsigned int parentTick; // this node's own parent (ABSOLUTE); (0, 0xFFFFFFFF) = root
+ unsigned int parentSolutionIndexInTick;
+ unsigned int score; // error count; a child must score strictly below this
+ unsigned int childCount; // children already attached (compare vs maxChildrenPerParent)
+ unsigned int anchorTick;
+ unsigned int depth;
+}
+```
+
+Paging every node of a pubkey reconstructs the whole tree, edges included, with no further fetches.
+
+**(c) Parent ANN** - `REQUEST_ANT_PARENT_ANN` (74) / `RESPOND_ANT_PARENT_ANN` (75). **Operator-signed.**
+
+Request `RequestAntParentAnn` (8 bytes) + 64-byte signature:
+
+```
+unsigned int parentRefTick;
+unsigned int parentRefSolutionIndexInTick;
+```
+
+Response `RespondAntParentAnnHeader` (16 bytes), then `annSizeBytes` of **canonical ANN** (one trit per
+byte, the exact form the scorer consumes - no unpacking needed):
+
+```
+unsigned int parentRefTick;
+unsigned int parentRefSolutionIndexInTick;
+unsigned int annSizeBytes; // ANN LUT size when status is OK, else 0
+unsigned char status; // 0 = OK, 1 = NOT_FOUND, 2 = IS_ROOT (derive your own root instead)
+unsigned char padding[3];
+```
+
+### 2.8 Network message + transaction types
+
+| Type | Value | Signed | Direction |
+|------|-------|--------|-----------|
+| `BROADCAST_MESSAGE` + `MESSAGE_TYPE_ANT_SOLUTION` | `3` | envelope-signed | miner -> computor (submit) |
+| `AntColonyMiningSolutionTransaction` (`inputType`) | `12` | computor-signed | computor -> tick data (consensus) |
+| `REQUEST_ANT_IDENTITY_TREE` / `RESPOND_ANT_IDENTITY_TREE` | `72` / `73` | operator | read tree |
+| `REQUEST_ANT_PARENT_ANN` / `RESPOND_ANT_PARENT_ANN` | `74` / `75` | operator | read one node's ANN |
+| `REQUEST_ANT_EPOCH_CONTEXT` / `RESPOND_ANT_EPOCH_CONTEXT` | `76` / `77` | public | read epoch params |
diff --git a/doc/protocol.md b/doc/protocol.md
index 70fcc0c84..851217582 100644
--- a/doc/protocol.md
+++ b/doc/protocol.md
@@ -29,6 +29,7 @@ The following transaction types (`tx->inputType`) are defined:
- `ExecutionFeeReportTransactionPrefix`, type 9, defined in `src/network_messages/execution_fees.h`.
- `OracleUserQueryTransactionPrefix`, type 10, defined in `src/oracle_core/oracle_transactions.h`.
- `DogeMiningShareTransaction`, type 11, defined in `src/mining/mining.h`.
+- `AntColonyMiningSolutionTransaction`, type 12, defined in `src/mining/mining.h`.
## Peer Sharing
diff --git a/src/Qubic.vcxproj b/src/Qubic.vcxproj
index 942c974b0..db0752908 100644
--- a/src/Qubic.vcxproj
+++ b/src/Qubic.vcxproj
@@ -66,15 +66,20 @@
+
+
+
+
+
diff --git a/src/Qubic.vcxproj.filters b/src/Qubic.vcxproj.filters
index 160c5365e..0c2a00709 100644
--- a/src/Qubic.vcxproj.filters
+++ b/src/Qubic.vcxproj.filters
@@ -50,6 +50,9 @@
network_messages
+
+ network_messages
+
network_messages
@@ -185,6 +188,18 @@
mining
+
+ mining\ant_colony
+
+
+ mining\ant_colony
+
+
+ mining\ant_colony
+
+
+ mining
+
logging
@@ -456,6 +471,9 @@
{df525479-7504-470c-a25a-de4af8be0e5d}
+
+ {7b1f3a52-9c4e-4d2a-b8e6-2a5c9d417f60}
+
{d334594b-f24d-440e-949a-c791aa13f867}
diff --git a/src/logging/logging.h b/src/logging/logging.h
index f8e389e6d..9d6a5d180 100644
--- a/src/logging/logging.h
+++ b/src/logging/logging.h
@@ -66,6 +66,7 @@ struct Peer;
#define CUSTOM_MESSAGE_OP_END_DISTRIBUTE_DIVIDENDS 6217575821008457285ULL //END_DDIV
#define CUSTOM_MESSAGE_OP_START_EPOCH 4850183582582395987ULL // STA_EPOC
#define CUSTOM_MESSAGE_OP_END_EPOCH 4850183582582591045ULL //END_EPOC
+#define CUSTOM_MESSAGE_ANT_SOLUTION 6146374810954124865ULL // ANT_SOLU
/*
* STRUCTS FOR LOGGING
*/
@@ -191,6 +192,23 @@ struct DummyCustomMessage
char _terminator; // Only data before "_terminator" are logged
};
+// On-chain outcome of one ant-colony solution transaction (accepted or rejected),
+// identified by its dedup key (sourcePublicKey, parentRef, nonce).
+struct AntSolutionLogMessage
+{
+ unsigned long long _type; // CUSTOM_MESSAGE_ANT_SOLUTION
+ m256i sourcePublicKey;
+ m256i nonce;
+ unsigned int parentTick;
+ unsigned int parentSolutionIndexInTick;
+ unsigned int anchorTick;
+ unsigned int score;
+ // ValidityResult of the commit
+ unsigned int result;
+
+ char _terminator; // Only data before "_terminator" are logged
+};
+
struct Burning
{
m256i sourcePublicKey;
diff --git a/src/mining/ant_colony/ant_colony.h b/src/mining/ant_colony/ant_colony.h
new file mode 100644
index 000000000..195eb36b8
--- /dev/null
+++ b/src/mining/ant_colony/ant_colony.h
@@ -0,0 +1,1609 @@
+#pragma once
+
+#include "platform/assert.h"
+#include "platform/concurrency.h"
+#include "platform/m256.h"
+#include "platform/memory.h"
+#include "platform/memory_util.h"
+#include "kangaroo_twelve.h"
+#include "platform/file_io.h"
+#include "contract_core/pre_qpi_def.h"
+#include "qpi/qpi.h"
+#include "qpi/impl/qpi_hash_map_impl.h"
+#include "public_settings.h"
+#include "mining/mining.h"
+#include "mining/trit_pack.h"
+
+// The keyed structures are twice the population they index.
+static constexpr unsigned long long ANT_DEDUP_SIZE = 2ULL * ANT_MAX_NODES_PER_EPOCH;
+// At most one key per record, and records are capped, so load stays at or below 50% and set() cannot fail.
+static constexpr unsigned long long ANT_CHILD_HEAD_BY_PARENT_SIZE = 2ULL * ANT_MAX_NODES_PER_EPOCH;
+static_assert(ANT_CHILD_HEAD_BY_PARENT_SIZE >= 2ULL * ANT_MAX_NODES_PER_EPOCH,
+ "child-head-by-parent map must stay at or below 50% load so its set() cannot fail");
+// One entry per identity holding a tree. Unlike the map above, nothing caps how many identities
+// submit, so this one CAN fill - commit() fails closed with RejectMinerIndexFull.
+static constexpr unsigned long long ANT_CHILD_HEAD_BY_MINER_SIZE = 2ULL * MAX_NUMBER_OF_MINERS;
+
+// How many ANN the epoch's harvest keeps. The target is the LUT with the best error, so this is
+// simply the lowest N scores of the epoch - not one per identity, and not tied to the ranking
+static constexpr unsigned int ANT_EXPORT_MAX_SOLUTIONS = NUMBER_OF_COMPUTORS;
+
+// Serial scratch for the save/load header (meta + anchor ring + export set) and the solution export.
+// In case of this grow to large, consider use the one in common buffer
+static constexpr unsigned long long ANT_SNAPSHOT_SCRATCH_BYTES = 2ULL * 1024 * 1024; // 2MB
+
+static constexpr unsigned int NO_SIBLING = 0xFFFFFFFFU;
+static constexpr unsigned int WORST_SCORE = 0xFFFFFFFFU;
+static constexpr long long ANT_INVALID_INDEX = -1;
+
+// Anchor digests for recent ticks, indexed by tick & (size - 1). Smallest power of two holding
+// 2*(N+1) entries so a lookup inside the freshness window can never be aliased by a newer tick.
+static constexpr unsigned int antAnchorRingSize(unsigned int window)
+{
+ unsigned int size = 1;
+ while (size < 2u * (window + 1u))
+ {
+ size <<= 1;
+ }
+ return size;
+}
+static constexpr unsigned int ANT_ANCHOR_RING_SIZE = antAnchorRingSize(ANT_PUBLISH_WINDOW_TICKS);
+static constexpr unsigned int ANT_ANCHOR_TICK_NONE = 0xFFFFFFFFU;
+
+// (tick, solutionIndexInTick), ABSOLUTE system tick plus the solution transaction's index in tick.
+// Every tick in the subsystem is absolute; slotOf() is the only place a tick becomes a tick-index offset.
+struct SolutionRef
+{
+ unsigned int tick; // ABSOLUTE system tick
+ unsigned int solutionIndexInTick;
+
+ bool operator==(const SolutionRef& other) const
+ {
+ return (tick == other.tick) && (solutionIndexInTick == other.solutionIndexInTick);
+ }
+
+ bool isRoot() const
+ {
+ return (tick == 0) && (solutionIndexInTick == 0xFFFFFFFFu);
+ }
+};
+
+// A root of all trees
+static constexpr SolutionRef ROOT_REF = { 0u, 0xFFFFFFFFu };
+
+// A solution is uniquely identified by (pubkey, parentRef, nonce).
+struct AntDedupKey
+{
+ m256i pubkey;
+ m256i nonce;
+ SolutionRef parentRef;
+
+ bool operator==(const AntDedupKey& other) const
+ {
+ return (pubkey == other.pubkey) && (nonce == other.nonce) && (parentRef == other.parentRef);
+ }
+};
+
+struct AntSolutionRecord
+{
+ m256i pubkey;
+ m256i nonce;
+ SolutionRef parentRef; // this solution's parent, or ROOT_REF
+ SolutionRef selfRef; // this solution's own address (ABSOLUTE tick inside)
+ unsigned int score; // error count, lower is better
+ unsigned int anchorTick; // ABSOLUTE. tick whose digest seeded the RNG
+ unsigned int depth; // a child of the root is depth 1; the root itself is never stored
+ unsigned int childAnnHash; // K12 of the canonical ANN at commit; digest-fold input
+ unsigned int annStateSlot; // index into the ANN pool; always equals the record index
+ unsigned int nextSiblingIdx; // next child of the same parent, NO_SIBLING terminates
+};
+static_assert(sizeof(AntSolutionRecord) == 104, "AntSolutionRecord unexpected padding");
+
+// ANN that will be saved for the epoch
+template
+struct AntExportSlotT
+{
+ m256i pubkey;
+ unsigned int score;
+ unsigned int depth;
+ PackedAnnT ann;
+};
+
+// slotOf(tick) -> the run of records committed in that tick, so findIndexBySolutionRef() resolves a
+// SolutionRef without scanning the store. The run is unbroken because the store is append-only and
+// a tick's solutions all commit while that tick is processed
+struct AntTickSlot
+{
+ unsigned int startIdx; // this tick's first record
+ unsigned int count; // records this tick produced; written last, so it gates the run
+};
+
+// Results and diagnostics
+enum ValidityResult
+{
+ Valid,
+ // Passed every rule but the store is full, so it was not recorded. Still a valid solution: its
+ // score is already folded into resourceTestingDigest, and the caller must refund and rank it.
+ ValidNotStored,
+ RejectParentNotRegistered,
+ RejectStale, // anchor in the future, or published more than N ticks after it
+ RejectWrongTree, // parent belongs to a different identity
+ RejectBelowThreshold, // score above the per-epoch error bound
+ RejectLeParent, // did not strictly beat the parent
+ RejectMaxChildrenPerParent, // the parent already holds ANT_MAX_CHILDREN_PER_PARENT children
+ RejectTickOutOfRange,
+ RejectReplay, // (pubkey, parentRef, nonce) already committed this epoch
+ RejectDedupFull,
+ RejectMinerIndexFull, // more than MAX_NUMBER_OF_MINERS identities hold a tree this epoch
+ RejectNonCanonicalNonce, // scorer refused the nonce; no score was produced
+};
+
+struct AntColonyDiagnostics
+{
+ unsigned long long rejectParentNotRegistered;
+ unsigned long long rejectStale;
+ unsigned long long rejectWrongTree;
+ unsigned long long rejectThreshold;
+ unsigned long long rejectLeParent;
+ unsigned long long rejectMaxChildren;
+ unsigned long long rejectTickOutOfRange;
+ unsigned long long rejectReplay;
+ unsigned long long rejectDedupFull;
+ unsigned long long rejectMinerIndexFull;
+ unsigned long long rejectNonCanonicalNonce;
+
+ unsigned long long acceptedSolutions;
+ unsigned long long acceptedNotStored;
+ unsigned long long treeDepthMax;
+ unsigned long long treeSizeCurrent;
+
+ void reset()
+ {
+ setMem(this, sizeof(*this), 0);
+ }
+
+ void appendLog(CHAR16* message) const
+ {
+ appendText(message, L"tree ");
+ appendNumber(message, treeSizeCurrent, TRUE);
+ appendText(message, L"/");
+ appendNumber(message, ANT_MAX_NODES_PER_EPOCH, TRUE);
+ appendText(message, L" depth ");
+ appendNumber(message, treeDepthMax, FALSE);
+ appendText(message, L" | accepted ");
+ appendNumber(message, acceptedSolutions, TRUE);
+ appendText(message, L" (not stored ");
+ appendNumber(message, acceptedNotStored, TRUE);
+ appendText(message, L")");
+
+ appendText(message, L" | rejected: parent ");
+ appendNumber(message, rejectParentNotRegistered, TRUE);
+ appendText(message, L", stale ");
+ appendNumber(message, rejectStale, TRUE);
+ appendText(message, L", wrongTree ");
+ appendNumber(message, rejectWrongTree, TRUE);
+ appendText(message, L", threshold ");
+ appendNumber(message, rejectThreshold, TRUE);
+ appendText(message, L", leParent ");
+ appendNumber(message, rejectLeParent, TRUE);
+ appendText(message, L", maxChildren ");
+ appendNumber(message, rejectMaxChildren, TRUE);
+ appendText(message, L", tickRange ");
+ appendNumber(message, rejectTickOutOfRange, TRUE);
+ appendText(message, L", replay ");
+ appendNumber(message, rejectReplay, TRUE);
+ appendText(message, L", dedupFull ");
+ appendNumber(message, rejectDedupFull, TRUE);
+ appendText(message, L", minerIndexFull ");
+ appendNumber(message, rejectMinerIndexFull, TRUE);
+ appendText(message, L", nonCanonicalNonce ");
+ appendNumber(message, rejectNonCanonicalNonce, TRUE);
+ }
+
+ void count(ValidityResult r)
+ {
+ switch (r)
+ {
+ case ValidityResult::RejectParentNotRegistered: rejectParentNotRegistered++; break;
+ case ValidityResult::RejectStale: rejectStale++; break;
+ case ValidityResult::RejectWrongTree: rejectWrongTree++; break;
+ case ValidityResult::RejectBelowThreshold: rejectThreshold++; break;
+ case ValidityResult::RejectLeParent: rejectLeParent++; break;
+ case ValidityResult::RejectMaxChildrenPerParent: rejectMaxChildren++; break;
+ case ValidityResult::RejectTickOutOfRange: rejectTickOutOfRange++; break;
+ case ValidityResult::RejectReplay: rejectReplay++; break;
+ case ValidityResult::RejectDedupFull: rejectDedupFull++; break;
+ case ValidityResult::RejectMinerIndexFull: rejectMinerIndexFull++; break;
+ case ValidityResult::RejectNonCanonicalNonce: rejectNonCanonicalNonce++; break;
+ default: break;
+ }
+ }
+};
+
+// A proposed child, reduced to what the admission rules read. Deliberately narrower than
+// AntCommitInput so validateChild() stays a pure predicate.
+struct ChildCandidate
+{
+ m256i pubkey;
+ unsigned int score; // error count, lower is better
+ unsigned int anchorTick; // ABSOLUTE
+ unsigned int publishTick; // ABSOLUTE
+};
+
+// Every tick here is an ABSOLUTE system tick: selfRef/parentRef ticks, anchorTick and publishTick all
+// share one basis (publishTick equals selfRef.tick), so no cross-basis comparison can slip in.
+struct AntCommitInput
+{
+ m256i pubkey;
+ m256i nonce;
+ SolutionRef parentRef;
+ SolutionRef selfRef;
+ unsigned int anchorTick; // ABSOLUTE
+ unsigned int publishTick; // ABSOLUTE
+};
+
+struct AnchorRing
+{
+ unsigned int ticks[ANT_ANCHOR_RING_SIZE];
+ m256i digests[ANT_ANCHOR_RING_SIZE];
+};
+
+// ---------------------------------------------------------------------------------------------
+
+template
+class AntColony
+{
+public:
+ // The ANN state will depend on score type
+ using Ann = typename ScoreT::ANN;
+
+ // In-store form of Ann: 2 bits per trit
+ using PackedAnn = score_engine::PackedTrits;
+ static_assert(sizeof(PackedAnn) == PackedAnn::groupCount * sizeof(unsigned long long),
+ "PackedAnn must not be padded");
+ // Catches sizing from a scorer's padded genome (bpp9000: lutSize 27 vs PaddedLut 32).
+ static_assert(PackedAnn::tritCount == sizeof(Ann), "PackedAnn must cover exactly one ANN");
+
+ using ExportSlot = AntExportSlotT;
+
+ // The epoch's best ANN, maintained as solutions arrive
+ struct ExportSet
+ {
+ ExportSlot slots[ANT_EXPORT_MAX_SOLUTIONS];
+ // Indices into slots, ascending by score. Kept separate so an insert shifts 4-byte indices
+ // rather than 552-byte slots.
+ unsigned int order[ANT_EXPORT_MAX_SOLUTIONS];
+ unsigned int count;
+ unsigned int padding;
+ };
+
+ static constexpr unsigned long long ANT_RECORDS_BYTES =
+ (unsigned long long)ANT_MAX_NODES_PER_EPOCH * sizeof(AntSolutionRecord);
+ static constexpr unsigned long long ANT_ANN_POOL_BYTES =
+ (unsigned long long)ANT_MAX_NODES_PER_EPOCH * sizeof(PackedAnn);
+
+ // The score is actually a function of below
+ struct ReplayKey
+ {
+ m256i pubkey;
+ m256i nonce;
+ m256i parentAnnHash; // K12 of the parent's ANN bytes; zero for a child of the root
+ m256i anchorDigest; // the digest the walk consumed, not the tick it came from
+
+ bool operator==(const ReplayKey& other) const
+ {
+ return (pubkey == other.pubkey) && (nonce == other.nonce)
+ && (parentAnnHash == other.parentAnnHash) && (anchorDigest == other.anchorDigest);
+ }
+ };
+ static_assert(sizeof(ReplayKey) == 4 * sizeof(m256i), "ReplayKey must be padding-free");
+
+ struct ReplayEntry
+ {
+ ReplayKey key;
+ PackedAnn ann;
+ unsigned int score;
+ unsigned int occupied;
+ };
+
+ // Padding-free, so the on-disk entry matches the in-memory one byte for byte.
+ static_assert(sizeof(ReplayEntry) ==
+ sizeof(ReplayKey) + sizeof(PackedAnn) + 2 * sizeof(unsigned int),
+ "ReplayEntry unexpected padding");
+
+ static constexpr unsigned long long ANT_REPLAY_CACHE_BYTES =
+ (unsigned long long)ANT_REPLAY_CACHE_SIZE * sizeof(ReplayEntry);
+
+ bool init();
+ void deinit();
+
+ // Wipe the whole tree. A new epoch starts empty and reseeded.
+ void reset();
+
+ void beginEpoch(const m256i& rootSeed, unsigned int initialTick)
+ {
+ reset();
+ clearReplayCache();
+ _rootSeed = rootSeed;
+ _initialTick = initialTick;
+ }
+
+ const m256i& rootSeed() const
+ {
+ return _rootSeed;
+ }
+
+ void setErrorThreshold(unsigned int t)
+ {
+ _errorThreshold = t;
+ }
+
+ unsigned int errorThreshold() const
+ {
+ return _errorThreshold;
+ }
+
+ unsigned int solutionCount() const
+ {
+ return _solutionCount;
+ }
+
+ // Slots a miner can still claim. Reaching zero does not stop acceptance, a valid solution is
+ // still scored, folded, refunded and ranked, but no NEW branch point can be created, which is
+ // what a miner needs to know before planning a lineage.
+ unsigned int freeAnnSlotsCount() const
+ {
+ return (_solutionCount < ANT_MAX_NODES_PER_EPOCH) ? (ANT_MAX_NODES_PER_EPOCH - _solutionCount) : 0;
+ }
+
+ const AntColonyDiagnostics& stats() const
+ {
+ return _stats;
+ }
+
+ void recordReject(ValidityResult r)
+ {
+ ASSERT(r != ValidityResult::Valid);
+ _stats.count(r);
+ }
+
+ // Only records, the ANN pool and the anchor ring are written;
+ // the tick index, both head maps and the dedup set are DERIVED and are
+ // rebuilt from the records on load
+ bool saveSnapshot(unsigned short epoch, CHAR16* directory, unsigned int initialTick) const;
+ // rootSeed and errorThreshold are the NODE's values, not the file's. The snapshot must agree
+ // with them or it is refused
+ bool loadSnapshot(unsigned short epoch, CHAR16* directory,
+ const m256i& rootSeed, unsigned int errorThreshold, unsigned int initialTick);
+
+ void putReplayScore(const ReplayKey& key, unsigned int score, const Ann& ann);
+ bool tryGetReplayScore(const ReplayKey& key, unsigned int& outScore, Ann& outAnn);
+ void clearReplayCache();
+ unsigned int replayCacheOccupancy() const
+ {
+ return _replayCacheOccupancy;
+ }
+ bool saveReplayCache(unsigned short epoch, CHAR16* directory);
+ bool loadReplayCache(unsigned short epoch, CHAR16* directory);
+
+ // Writes the ANT_EXPORT_MAX_SOLUTIONS lowest-scoring networks of the epoch to a file for offline
+ // extraction. MUST be called between endEpoch() and the reset that starts the next epoch
+ bool exportBestSolutions(unsigned short epoch, CHAR16* directory);
+
+ // Children already recorded under this parent, capped at ANT_MAX_CHILDREN_PER_PARENT, for query
+ // purposes. Off-thread safe: the head map is read under the lock, the chain walk after it is not.
+ unsigned int childCountForQuery(const SolutionRef& parentRef, const m256i& childPubkey)
+ {
+ unsigned int head = NO_SIBLING;
+ // Take the latest child first with lock to touch the head map
+ {
+ LockGuard guard(_headMapLock);
+ if (!chainHead(parentRef, childPubkey, head))
+ {
+ return 0;
+ }
+ }
+
+ // Escape the lock, then count from head, in which the record is imutable
+ return childCountFromHead(head);
+ }
+
+ // Anchor digests. Both take an ABSOLUTE system tick.
+ // Called from tick processor only
+ void recordAnchorDigest(unsigned int tick, const m256i& digest);
+ // Can be called from any processors
+ bool getAnchorDigest(unsigned int tick, m256i& digest) const;
+
+ // Tree access
+
+ // nullptr when idx is out of range
+ const AntSolutionRecord* recordAt(long long idx) const
+ {
+ if (idx < 0 || (unsigned long long)idx >= _solutionCount)
+ {
+ return nullptr;
+ }
+ return &_records[idx];
+ }
+
+ // Unpacks a stored network into the caller's buffer. ROOT is never a record, so callers must
+ // handle parentRef.isRoot() before reaching here.
+ bool annOfNonRoot(const AntSolutionRecord& rec, Ann& out) const
+ {
+ if (rec.annStateSlot >= _solutionCount)
+ {
+ return false;
+ }
+ _annPool[rec.annStateSlot].unpack(out.lut);
+ return true;
+ }
+
+ long long findIndexBySolutionRef(const SolutionRef& ref) const;
+
+ // Constraint specific functions
+
+ // Resolves a parent for scoring. outParentRec is null for ROOT_REF, the caller derives the
+ // per-identity root from the submitter's pubkey instead.
+ ValidityResult tryGetParent(const SolutionRef& parentRef,
+ const AntSolutionRecord** outParentRec) const;
+
+ // Admission rules for a proposed child: freshness, tree ownership, threshold, parent, per-parent
+ // child cap. Static and pure, so the rule set is testable without a colony. Lower score is better.
+ static ValidityResult validateChild(const ChildCandidate& child,
+ const AntSolutionRecord* parentRecord, unsigned int childCount, unsigned int threshold);
+
+ // Validates and, if accepted, appends the record and its network to the store.
+ ValidityResult commit(const AntCommitInput& in, const AntSolutionRecord* parentRec,
+ unsigned int score, const Ann& childAnn, unsigned int childAnnHash);
+
+private:
+ // Children already recorded under this parent, capped at ANT_MAX_CHILDREN_PER_PARENT. Root
+ // children are keyed by miner, deeper ones by parent.
+ unsigned int countChildren(const SolutionRef& parentRef, const m256i& childPubkey) const;
+
+ // The map read, and the ONLY part of a child count that touches a head map. Split out because the
+ // walk that follows it does not, which is what lets the query hold the lock for a constant time
+ // instead of a whole chain.
+ // Depth-1 nodes chain per identity; deeper nodes chain per parent, which is single-identity by the
+ // wrong-tree check.
+ bool chainHead(const SolutionRef& parentRef, const m256i& childPubkey, unsigned int& out) const
+ {
+ if (parentRef.isRoot())
+ {
+ return _childHeadByMiner->get(childPubkey, out);
+ }
+ return _childHeadByParent->get(parentRef, out);
+ }
+
+ // The walk half, from a chain head already in hand. Takes no lock: _records is append-only and a
+ // record's nextSiblingIdx is written once at commit and never touched again, so a chain is stable
+ // to follow even while the tick processor is appending elsewhere.
+ unsigned int childCountFromHead(unsigned int head) const;
+
+ // Offers a solution to the epoch's best-N set. Called for EVERY solution that passes the rules
+ void noteExportCandidate(const m256i& pubkey, unsigned int score, unsigned int depth, const Ann& ann)
+ {
+ ExportSet& set = *_exportSet;
+ unsigned int slot;
+ if (set.count < ANT_EXPORT_MAX_SOLUTIONS)
+ {
+ slot = set.count;
+ }
+ else if (score >= set.slots[set.order[ANT_EXPORT_MAX_SOLUTIONS - 1]].score)
+ {
+ // The common case once the set is full
+ return;
+ }
+ else
+ {
+ // Reuse the worst entry's storage; its index leaves the order below.
+ slot = set.order[ANT_EXPORT_MAX_SOLUTIONS - 1];
+ }
+
+ set.slots[slot].pubkey = pubkey;
+ set.slots[slot].score = score;
+ set.slots[slot].depth = depth;
+ set.slots[slot].ann.pack(ann.lut);
+
+ // Insert into the order, shifting indices only. Equal scores keep the incumbent ahead, so
+ // among equals the earlier solution ranks first
+ const unsigned int end = (set.count < ANT_EXPORT_MAX_SOLUTIONS) ? set.count : (ANT_EXPORT_MAX_SOLUTIONS - 1);
+ unsigned int i = end;
+ while (i > 0 && set.slots[set.order[i - 1]].score > score)
+ {
+ set.order[i] = set.order[i - 1];
+ i--;
+ }
+ set.order[i] = slot;
+ if (set.count < ANT_EXPORT_MAX_SOLUTIONS)
+ {
+ set.count++;
+ }
+ }
+
+ // loadSnapshot() helper: rebuild the tick index, head maps and dedup set from the loaded
+ // records, treating them as untrusted input. Uses _initialTick, set by the caller beforehand.
+ bool rebuildDerivedState();
+
+ static unsigned int replaySlotOf(const ReplayKey& key)
+ {
+ unsigned long long digest;
+ KangarooTwelve(&key, sizeof(key), &digest, sizeof(digest));
+ return (unsigned int)(digest & (ANT_REPLAY_CACHE_SIZE - 1));
+ }
+
+ // The sole place an absolute tick becomes a tick-index offset. Returns false when the tick falls
+ // outside this epoch's window (before initialTick, or past the per-epoch tick cap); callers treat
+ // that as "no such record" - RejectParentNotRegistered on lookup, RejectTickOutOfRange on commit.
+ bool slotOf(unsigned int tick, unsigned int& slot) const
+ {
+ if (tick < _initialTick)
+ {
+ return false;
+ }
+ slot = tick - _initialTick;
+ return slot < MAX_NUMBER_OF_TICKS_PER_EPOCH;
+ }
+
+ AntSolutionRecord* _records;
+ PackedAnn* _annPool;
+ AntTickSlot* _tickIndex;
+ AnchorRing* _anchors;
+
+ // Solutions already committed this epoch, so a resend is rejected instead of re-added.
+ QPI::HashSet* _dedup;
+ ExportSet* _exportSet;
+
+ // The two head maps have no reader/writer protocol of their own: QPI::HashMap::set() makes a
+ // slot's key visible before its value, so a reader asking for the key being inserted can get a
+ // garbage index
+ volatile char _headMapLock;
+ // Both give countChildren() a parent's children without scanning the store: the value is the
+ // newest child's record index, and nextSiblingIdx chains back to the older ones.
+ // parent's address -> newest child
+ QPI::HashMap* _childHeadByParent;
+ // miner's pubkey -> newest depth-1 node (a child OF that miner's root, not a root itself).
+ // Keyed by miner because ROOT_REF is shared by everyone.
+ QPI::HashMap* _childHeadByMiner;
+
+ ReplayEntry* _replayCache;
+ volatile char _replayCacheLock;
+
+ // Serial scratch for save/load and the solution export
+ unsigned char* _snapshotScratch;
+ unsigned int _replayCacheOccupancy;
+
+ unsigned int _solutionCount;
+ unsigned int _errorThreshold;
+ // This epoch's first tick. slotOf() maps an absolute tick to a tick-index offset against it.
+ unsigned int _initialTick;
+ m256i _rootSeed;
+ AntColonyDiagnostics _stats;
+};
+
+template
+inline bool AntColony::init()
+{
+ setMem(this, sizeof(*this), 0);
+
+ if (!allocPoolWithErrorLog(L"AntColony::_records",
+ ANT_RECORDS_BYTES, (void**)&_records, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_annPool",
+ ANT_ANN_POOL_BYTES, (void**)&_annPool, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_tickIndex",
+ (unsigned long long)MAX_NUMBER_OF_TICKS_PER_EPOCH * sizeof(AntTickSlot),
+ (void**)&_tickIndex, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_anchors",
+ sizeof(AnchorRing), (void**)&_anchors, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_childHeadByParent",
+ sizeof(QPI::HashMap),
+ (void**)&_childHeadByParent, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_childHeadByMiner",
+ sizeof(QPI::HashMap),
+ (void**)&_childHeadByMiner, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_exportSet",
+ sizeof(ExportSet), (void**)&_exportSet, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_dedup",
+ sizeof(QPI::HashSet),
+ (void**)&_dedup, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_replayCache",
+ ANT_REPLAY_CACHE_BYTES, (void**)&_replayCache, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntColony::_snapshotScratch",
+ ANT_SNAPSHOT_SCRATCH_BYTES, (void**)&_snapshotScratch, __LINE__))
+ {
+ return false;
+ }
+
+ reset();
+ clearReplayCache();
+ return true;
+}
+
+template
+inline void AntColony::deinit()
+{
+ if (_snapshotScratch)
+ {
+ freePool(_snapshotScratch);
+ }
+ if (_replayCache)
+ {
+ freePool(_replayCache);
+ }
+ if (_exportSet)
+ {
+ freePool(_exportSet);
+ }
+ if (_dedup)
+ {
+ freePool(_dedup);
+ }
+ if (_childHeadByMiner)
+ {
+ freePool(_childHeadByMiner);
+ }
+ if (_childHeadByParent)
+ {
+ freePool(_childHeadByParent);
+ }
+ if (_anchors)
+ {
+ freePool(_anchors);
+ }
+ if (_tickIndex)
+ {
+ freePool(_tickIndex);
+ }
+ if (_annPool)
+ {
+ freePool(_annPool);
+ }
+ if (_records)
+ {
+ freePool(_records);
+ }
+
+ _replayCache = nullptr;
+ _exportSet = nullptr;
+ _dedup = nullptr;
+ _childHeadByMiner = nullptr;
+ _childHeadByParent = nullptr;
+ _anchors = nullptr;
+ _tickIndex = nullptr;
+ _annPool = nullptr;
+ _records = nullptr;
+}
+
+template
+inline void AntColony::reset()
+{
+ ASSERT(_records != nullptr);
+ ASSERT(_annPool != nullptr);
+ ASSERT(_tickIndex != nullptr);
+ ASSERT(_anchors != nullptr);
+ ASSERT(_childHeadByParent != nullptr);
+ ASSERT(_childHeadByMiner != nullptr);
+ ASSERT(_dedup != nullptr);
+ ASSERT(_exportSet != nullptr);
+
+ setMem(_records, ANT_RECORDS_BYTES, 0);
+ setMem(_tickIndex,
+ (unsigned long long)MAX_NUMBER_OF_TICKS_PER_EPOCH * sizeof(AntTickSlot), 0);
+ _childHeadByParent->reset();
+ _childHeadByMiner->reset();
+ _dedup->reset();
+ setMem(_exportSet, sizeof(ExportSet), 0);
+
+ // ANT_ANCHOR_TICK_NONE is used rather than zero
+ for (unsigned int i = 0; i < ANT_ANCHOR_RING_SIZE; i++)
+ {
+ _anchors->ticks[i] = ANT_ANCHOR_TICK_NONE;
+ _anchors->digests[i] = m256i::zero();
+ }
+
+ _solutionCount = 0;
+ _rootSeed = m256i::zero();
+
+ // Forgets setErrorThreshold rejects everything but a perfect score, rather than silently reusing the previous epoch's bound.
+ _errorThreshold = 0;
+
+ _stats.reset();
+}
+
+template
+inline void AntColony::clearReplayCache()
+{
+ if (_replayCache == nullptr)
+ {
+ return;
+ }
+ LockGuard guard(_replayCacheLock);
+ for (unsigned int i = 0; i < ANT_REPLAY_CACHE_SIZE; i++)
+ {
+ _replayCache[i].occupied = 0;
+ }
+ _replayCacheOccupancy = 0;
+}
+
+template
+inline void AntColony::putReplayScore(const ReplayKey& key, unsigned int score, const Ann& ann)
+{
+ if (_replayCache == nullptr)
+ {
+ return;
+ }
+ ReplayEntry staged;
+ staged.key = key;
+ staged.ann.pack(ann.lut);
+ staged.score = score;
+ staged.occupied = 1;
+
+ ReplayEntry& slot = _replayCache[replaySlotOf(key)];
+ LockGuard guard(_replayCacheLock);
+ if (!slot.occupied)
+ {
+ _replayCacheOccupancy++;
+ }
+ copyMem(&slot, &staged, sizeof(ReplayEntry));
+}
+
+template
+inline bool AntColony::tryGetReplayScore(const ReplayKey& key, unsigned int& outScore, Ann& outAnn)
+{
+ if (_replayCache == nullptr)
+ {
+ return false;
+ }
+ ReplayEntry& slot = _replayCache[replaySlotOf(key)];
+ LockGuard guard(_replayCacheLock);
+ if (!slot.occupied || !(slot.key == key))
+ {
+ return false;
+ }
+ outScore = slot.score;
+ slot.ann.unpack(outAnn.lut);
+ return true;
+}
+
+// Cache the already computed score for the ant colony
+
+template
+inline bool AntColony::saveReplayCache(unsigned short epoch, CHAR16* directory)
+{
+ if (_replayCache == nullptr)
+ {
+ return false;
+ }
+ addEpochToFileName(ANT_COLONY_REPLAY_CACHE_FILENAME,
+ sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME) / sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME[0]), epoch);
+
+ // Held across the file IO so the table cannot change under the write. That blocks the solution
+ // processors for the duration
+ LockGuard guard(_replayCacheLock);
+ if (saveLargeFile(ANT_COLONY_REPLAY_CACHE_FILENAME, ANT_REPLAY_CACHE_BYTES,
+ (unsigned char*)_replayCache, directory, false) != (long long)ANT_REPLAY_CACHE_BYTES)
+ {
+ logToConsole(L"[ant-colony] failed to save replay cache");
+ return false;
+ }
+ return true;
+}
+
+template
+inline bool AntColony::loadReplayCache(unsigned short epoch, CHAR16* directory)
+{
+ if (_replayCache == nullptr)
+ {
+ return false;
+ }
+ addEpochToFileName(ANT_COLONY_REPLAY_CACHE_FILENAME,
+ sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME) / sizeof(ANT_COLONY_REPLAY_CACHE_FILENAME[0]), epoch);
+
+ LockGuard guard(_replayCacheLock);
+ if (loadLargeFile(ANT_COLONY_REPLAY_CACHE_FILENAME, ANT_REPLAY_CACHE_BYTES,
+ (unsigned char*)_replayCache, directory) != (long long)ANT_REPLAY_CACHE_BYTES)
+ {
+ // Absent at the start of an epoch, and a wrong size means another build wrote it. Either way
+ // the table is zeroed and every solution gets computed honestly.
+ logToConsole(L"[ant-colony] no usable replay cache, solutions will be recomputed");
+ setMem(_replayCache, ANT_REPLAY_CACHE_BYTES, 0);
+ _replayCacheOccupancy = 0;
+ return false;
+ }
+
+ unsigned int occupied = 0;
+ for (unsigned int i = 0; i < ANT_REPLAY_CACHE_SIZE; i++)
+ {
+ if (_replayCache[i].occupied)
+ {
+ occupied++;
+ }
+ }
+ _replayCacheOccupancy = occupied;
+
+ CHAR16 message[192];
+ setText(message, L"[ant-colony] replay cache loaded, entries ");
+ appendNumber(message, occupied, FALSE);
+ logToConsole(message);
+ return true;
+}
+
+// The epoch's harvest file
+
+
+// Written once at the front of the file
+struct AntColonyExportHeader
+{
+ unsigned int epoch;
+ unsigned int entryCount;
+ unsigned int entrySizeBytes; // of AntColonyExportEntry, not of a store record
+ unsigned int annSizeBytes;
+ unsigned int solutionCount; // the whole epoch, of which entryCount are exported
+ unsigned int errorThreshold;
+ unsigned char topologyHash[32]; // == BPP9000_TOPOLOGY_HASH of the build that wrote this
+ unsigned char dataHash[32]; // == BPP9000_DATA_HASH
+ m256i rootSeed;
+};
+static_assert(sizeof(AntColonyExportHeader) == 24 + 64 + 32, "AntColonyExportHeader unexpected padding");
+
+// What the harvest needs to reproduce the best error
+struct AntColonyExportEntry
+{
+ // Names the identity that found this network, log only
+ m256i pubkey;
+ unsigned int score; // error count, lower is better - how good this network is
+ unsigned int depth; // generations of strict improvement behind it, so the chain length is visible
+};
+static_assert(sizeof(AntColonyExportEntry) == 32 + 8, "AntColonyExportEntry unexpected padding");
+
+template
+inline bool AntColony::exportBestSolutions(unsigned short epoch, CHAR16* directory)
+{
+ ASSERT(_exportSet != nullptr);
+
+ struct Entry
+ {
+ AntColonyExportEntry meta;
+ Ann ann;
+ };
+ const ExportSet& set = *_exportSet;
+
+ AntColonyExportHeader header;
+ setMem(&header, sizeof(header), 0);
+ header.epoch = epoch;
+ header.entryCount = set.count;
+ header.entrySizeBytes = (unsigned int)sizeof(AntColonyExportEntry);
+ header.annSizeBytes = (unsigned int)sizeof(Ann);
+ header.solutionCount = _solutionCount;
+ header.errorThreshold = _errorThreshold;
+ copyMem(header.topologyHash, BPP9000_TOPOLOGY_HASH, sizeof(header.topologyHash));
+ copyMem(header.dataHash, BPP9000_DATA_HASH, sizeof(header.dataHash));
+ header.rootSeed = _rootSeed;
+
+ if (set.count == 0)
+ {
+ return save(ANT_COLONY_SOLUTIONS_EOE_FILENAME, sizeof(header), (unsigned char*)&header, directory)
+ == (long long)sizeof(header);
+ }
+
+ static_assert(sizeof(AntColonyExportHeader) + (unsigned long long)ANT_EXPORT_MAX_SOLUTIONS * sizeof(Entry)
+ <= ANT_SNAPSHOT_SCRATCH_BYTES, "ant solution export exceeds the scratch buffer");
+ const unsigned long long totalBytes = sizeof(header) + (unsigned long long)set.count * sizeof(Entry);
+ unsigned char* buffer = _snapshotScratch;
+
+ copyMem(buffer, &header, sizeof(header));
+ Entry* out = (Entry*)(buffer + sizeof(header));
+ for (unsigned int i = 0; i < set.count; i++)
+ {
+ const ExportSlot& slot = set.slots[set.order[i]];
+ out[i].meta.pubkey = slot.pubkey;
+ out[i].meta.score = slot.score;
+ out[i].meta.depth = slot.depth;
+ slot.ann.unpack(out[i].ann.lut);
+ }
+
+ const long long saved = save(ANT_COLONY_SOLUTIONS_EOE_FILENAME, totalBytes, buffer, directory);
+
+ if (saved != (long long)totalBytes)
+ {
+ logToConsole(L"[ant-colony] failed to write the solution export");
+ return false;
+ }
+
+ CHAR16 message[192];
+ setText(message, L"[ant-colony] exported best networks, entries ");
+ appendNumber(message, set.count, FALSE);
+ appendText(message, L", best error ");
+ appendNumber(message, set.slots[set.order[0]].score, FALSE);
+ appendText(message, L", worst kept ");
+ appendNumber(message, set.slots[set.order[set.count - 1]].score, FALSE);
+ logToConsole(message);
+ return true;
+}
+
+template
+inline void AntColony::recordAnchorDigest(unsigned int tick, const m256i& digest)
+{
+ const unsigned int slot = tick & (ANT_ANCHOR_RING_SIZE - 1);
+ // Invalidate first
+ ATOMIC_STORE32(_anchors->ticks[slot], (long)ANT_ANCHOR_TICK_NONE);
+ _anchors->digests[slot] = digest;
+ // The tick marker is written last, a reader that sees the tick must already see its digest.
+ ATOMIC_STORE32(_anchors->ticks[slot], (long)tick);
+}
+
+template
+inline bool AntColony::getAnchorDigest(unsigned int tick, m256i& digest) const
+{
+ if (tick == ANT_ANCHOR_TICK_NONE)
+ {
+ return false;
+ }
+ const unsigned int slot = tick & (ANT_ANCHOR_RING_SIZE - 1);
+ // Seqlock read, the tick is loaded atomically before and after the digest copy, so a re-check
+ // that still sees the same tick means the digest was not overwritten mid-copy.
+ if ((unsigned int)ATOMIC_LOAD32(_anchors->ticks[slot]) != tick)
+ {
+ return false; // never recorded, or aged out and overwritten by a newer tick
+ }
+ digest = _anchors->digests[slot];
+ return ((unsigned int)ATOMIC_LOAD32(_anchors->ticks[slot]) == tick);
+}
+
+template
+inline long long AntColony::findIndexBySolutionRef(const SolutionRef& ref) const
+{
+ unsigned int tickSlot = 0;
+ if (ref.isRoot() || !slotOf(ref.tick, tickSlot))
+ {
+ return ANT_INVALID_INDEX;
+ }
+ // Start from tick' begin index in the record and loop total of record in the tick
+ const AntTickSlot& slot = _tickIndex[tickSlot];
+ for (unsigned int i = 0; i < slot.count; i++)
+ {
+ const unsigned int idx = slot.startIdx + i;
+ if (idx >= _solutionCount)
+ {
+ break;
+ }
+ if (_records[idx].selfRef == ref)
+ {
+ return (long long)idx;
+ }
+ }
+ return ANT_INVALID_INDEX;
+}
+
+template
+inline unsigned int AntColony::countChildren(const SolutionRef& parentRef, const m256i& childPubkey) const
+{
+ // Tick processor only, so the head map read needs no lock here - nothing else writes it.
+ unsigned int head = NO_SIBLING;
+ if (!chainHead(parentRef, childPubkey, head))
+ {
+ return 0;
+ }
+ return childCountFromHead(head);
+}
+
+template
+inline unsigned int AntColony::childCountFromHead(unsigned int head) const
+{
+ // Count only up to the cap: past it the child is rejected regardless, so the walk and with it
+ // the whole per-commit cost, is bounded by ANT_MAX_CHILDREN_PER_PARENT. A cap of 0 (unbound)
+ // leaves the loop empty and returns 0, since the count is then never used.
+ unsigned int count = 0;
+ unsigned int idx = head;
+ while (idx != NO_SIBLING && count < ANT_MAX_CHILDREN_PER_PARENT)
+ {
+ // NOT a redundant bounds check - it is the publication barrier this walk relies on. commit()
+ // points the head map at a new index BEFORE it writes that record, and only bumps
+ // _solutionCount once the record is complete. An off-thread walk that reaches the new index
+ // early therefore stops here instead of reading half a record. Removing this reintroduces a
+ // torn read that would only ever show up under load.
+ if (idx >= _solutionCount)
+ {
+ break;
+ }
+ count++;
+ idx = _records[idx].nextSiblingIdx;
+ }
+ return count;
+}
+
+template
+inline ValidityResult AntColony::tryGetParent(const SolutionRef& parentRef,
+ const AntSolutionRecord** outParentRec) const
+{
+ *outParentRec = nullptr;
+ if (parentRef.isRoot())
+ {
+ return ValidityResult::Valid; // root is not a record; a null parent is the valid answer
+ }
+
+ const long long parentIdx = findIndexBySolutionRef(parentRef);
+ if (parentIdx == ANT_INVALID_INDEX)
+ {
+ return ValidityResult::RejectParentNotRegistered;
+ }
+ const AntSolutionRecord* rec = recordAt(parentIdx);
+ if (rec == nullptr)
+ {
+ return ValidityResult::RejectParentNotRegistered;
+ }
+ *outParentRec = rec;
+ return ValidityResult::Valid;
+}
+
+template
+inline ValidityResult AntColony::validateChild(const ChildCandidate& child,
+ const AntSolutionRecord* parentRecord, unsigned int childCount, unsigned int threshold)
+{
+ // Freshness, the anchor cannot be in the future, and publication cannot lag it by more than N.
+ if (child.anchorTick > child.publishTick
+ || (child.publishTick - child.anchorTick) > ANT_PUBLISH_WINDOW_TICKS)
+ {
+ return ValidityResult::RejectStale;
+ }
+
+ // A null parent record means ROOT, it has no score of its own, so seed WORST_SCORE and any child
+ // improves on it. A non-root parent must belong to the same identity
+ unsigned int parentScore = WORST_SCORE;
+ if (parentRecord != nullptr)
+ {
+ if (!(parentRecord->pubkey == child.pubkey))
+ {
+ return ValidityResult::RejectWrongTree;
+ }
+ parentScore = parentRecord->score;
+ }
+
+ if (child.score > threshold)
+ {
+ return ValidityResult::RejectBelowThreshold;
+ }
+ if (child.score >= parentScore)
+ {
+ return ValidityResult::RejectLeParent;
+ }
+ // Per-parent breadth cap. 0 means unbound - no cap.
+ if (ANT_MAX_CHILDREN_PER_PARENT != 0 && childCount >= ANT_MAX_CHILDREN_PER_PARENT)
+ {
+ return ValidityResult::RejectMaxChildrenPerParent;
+ }
+ return ValidityResult::Valid;
+}
+
+template
+inline ValidityResult AntColony::commit(const AntCommitInput& in, const AntSolutionRecord* parentRec,
+ unsigned int score, const Ann& childAnn, unsigned int childAnnHash)
+{
+ const unsigned int childCount = countChildren(in.parentRef, in.pubkey);
+ const ChildCandidate child{ in.pubkey, score, in.anchorTick, in.publishTick };
+
+ const ValidityResult result = validateChild(child, parentRec, childCount, _errorThreshold);
+ if (result != ValidityResult::Valid)
+ {
+ recordReject(result);
+ return result;
+ }
+
+ const AntDedupKey dedupKey{ in.pubkey, in.nonce, in.parentRef };
+ if (_dedup->contains(dedupKey))
+ {
+ recordReject(ValidityResult::RejectReplay);
+ return ValidityResult::RejectReplay;
+ }
+ // Store full. Every rule above already passed, so the solution is honest work and its score is
+ // in the digest whatever happens here - rejecting it would burn the deposit for a valid answer.
+ // Honour it and stop storing: the tree freezes, the leaderboard does not.
+ if (_solutionCount >= ANT_MAX_NODES_PER_EPOCH)
+ {
+ // The record is dropped, the network is not, still note this sols for end of epoch exppot
+ noteExportCandidate(in.pubkey, score, (parentRec != nullptr) ? (parentRec->depth + 1) : 1, childAnn);
+ _stats.acceptedNotStored++;
+ return ValidityResult::ValidNotStored;
+ }
+ unsigned int selfSlot = 0;
+ if (!slotOf(in.selfRef.tick, selfSlot))
+ {
+ recordReject(ValidityResult::RejectTickOutOfRange);
+ return ValidityResult::RejectTickOutOfRange;
+ }
+ // never commit a solution without recording its replay key. Cannot fire under the
+ // cap (population <= ANT_MAX_NODES_PER_EPOCH = 50% of ANT_DEDUP_SIZE), kept as a defensive check
+ if (_dedup->add(dedupKey) == QPI::NULL_INDEX)
+ {
+ recordReject(ValidityResult::RejectDedupFull);
+ return ValidityResult::RejectDedupFull;
+ }
+
+ const unsigned int newIdx = _solutionCount;
+
+ // Claim the sibling-chain head before writing the record
+ unsigned int prevHead = NO_SIBLING;
+ bool headClaimed = true;
+ {
+ LockGuard guard(_headMapLock);
+ if (in.parentRef.isRoot())
+ {
+ _childHeadByMiner->get(in.pubkey, prevHead);
+ headClaimed = (_childHeadByMiner->set(in.pubkey, newIdx) != QPI::NULL_INDEX);
+ }
+ else
+ {
+ _childHeadByParent->get(in.parentRef, prevHead);
+ _childHeadByParent->set(in.parentRef, newIdx); // cannot fail, see the static_assert on its size
+ }
+ }
+ if (!headClaimed)
+ {
+ // Fail closed. Degrading instead, accepting the node but leaving the identity without a
+ // chain head, would leave its children uncounted and silently disable its cap. Released
+ // first: this touches _dedup, which must not be reached under the head-map lock.
+ _dedup->remove(dedupKey);
+ recordReject(ValidityResult::RejectMinerIndexFull);
+ return ValidityResult::RejectMinerIndexFull;
+ }
+
+ // The record and its network share an index, which keeps the used portion of the allocation a
+ // contiguous prefix.
+ _annPool[newIdx].pack(childAnn.lut);
+
+ AntSolutionRecord& newRec = _records[newIdx];
+ newRec.pubkey = in.pubkey;
+ newRec.nonce = in.nonce;
+ newRec.parentRef = in.parentRef;
+ newRec.selfRef = in.selfRef;
+ newRec.score = score;
+ newRec.anchorTick = in.anchorTick;
+ newRec.depth = (parentRec != nullptr) ? (parentRec->depth + 1) : 1;
+ newRec.childAnnHash = childAnnHash;
+ newRec.annStateSlot = newIdx;
+ newRec.nextSiblingIdx = prevHead;
+
+ AntTickSlot& tslot = _tickIndex[selfSlot];
+ if (tslot.count == 0)
+ {
+ tslot.startIdx = newIdx;
+ }
+
+ // PUBLICATION ORDER, load-bearing. Readers on other threads gate on tslot.count, so everything
+ // they may then read must already be visible: record fields, then _solutionCount, then
+ // tslot.count last. _solutionCount must rise before tslot.count or findIndexBySolutionRef can
+ // resolve an index that recordAt() rejects.
+ // ATOMIC_STORE32 is here for the ordering barrier, not for atomicity of the value: these are
+ // plain unsigned ints written only by the tick processor
+ ATOMIC_STORE32(_solutionCount, (long)(newIdx + 1));
+ ATOMIC_STORE32(tslot.count, (long)(tslot.count + 1));
+
+ noteExportCandidate(in.pubkey, score, newRec.depth, childAnn);
+
+ _stats.acceptedSolutions++;
+ _stats.treeSizeCurrent = _solutionCount;
+ if (newRec.depth > _stats.treeDepthMax)
+ {
+ _stats.treeDepthMax = newRec.depth;
+ }
+ return ValidityResult::Valid;
+}
+
+// Only what cannot be derived is written. The tick index, both head maps and the dedup set are
+// rebuilt from the records
+
+struct AntColonySnapshotMeta
+{
+ unsigned int magic;
+ unsigned int version;
+ unsigned int epoch;
+ unsigned int solutionCount;
+ // Layout guards. A snapshot written by a build with a different record or ANN size must be
+ // refused rather than reinterpreted
+ unsigned int recordSizeBytes;
+ unsigned int annPoolEntryBytes;
+ unsigned int errorThreshold;
+ // This epoch's first tick. Records hold absolute ticks; the base must still match so slotOf() maps
+ // them into this node's tick index, and a snapshot from another epoch is refused rather than mis-read.
+ unsigned int initialTick;
+ unsigned int anchorRingBytes;
+ unsigned int exportSetBytes;
+ m256i rootSeed;
+
+ static constexpr unsigned int MAGIC = 0x414E5443; // "ANTC"
+ static constexpr unsigned int VERSION = 1; // SolutionRef holds absolute ticks
+};
+static_assert(sizeof(AntColonySnapshotMeta) == 40 + 32, "AntColonySnapshotMeta unexpected padding");
+
+static void antSnapshotFailure(const CHAR16* what, unsigned long long a, unsigned long long b)
+{
+ CHAR16 message[256];
+ setText(message, L"[ant-colony] snapshot: ");
+ appendText(message, what);
+ appendText(message, L" ");
+ appendNumber(message, a, FALSE);
+ appendText(message, L" / ");
+ appendNumber(message, b, FALSE);
+ logToConsole(message);
+}
+
+// Records and pool are sized by this, never by the raw count. At least one slot is always written,
+// so no snapshot file is ever zero length
+static unsigned long long antSnapshotSlotCount(unsigned int solutionCount)
+{
+ return (solutionCount > 0) ? (unsigned long long)solutionCount : 1ULL;
+}
+
+static void antSnapshotNameForEpoch(unsigned short epoch)
+{
+ addEpochToFileName(ANT_SNAPSHOT_HEADER_FILENAME, sizeof(ANT_SNAPSHOT_HEADER_FILENAME) / sizeof(ANT_SNAPSHOT_HEADER_FILENAME[0]), epoch);
+ addEpochToFileName(ANT_SNAPSHOT_RECORDS_FILENAME, sizeof(ANT_SNAPSHOT_RECORDS_FILENAME) / sizeof(ANT_SNAPSHOT_RECORDS_FILENAME[0]), epoch);
+ addEpochToFileName(ANT_SNAPSHOT_POOL_FILENAME, sizeof(ANT_SNAPSHOT_POOL_FILENAME) / sizeof(ANT_SNAPSHOT_POOL_FILENAME[0]), epoch);
+}
+
+template
+inline bool AntColony::saveSnapshot(unsigned short epoch, CHAR16* directory,
+ unsigned int initialTick) const
+{
+ ASSERT(_records != nullptr);
+ ASSERT(_annPool != nullptr);
+ ASSERT(_anchors != nullptr);
+
+ antSnapshotNameForEpoch(epoch);
+
+ AntColonySnapshotMeta meta;
+ setMem(&meta, sizeof(meta), 0);
+ meta.magic = AntColonySnapshotMeta::MAGIC;
+ meta.version = AntColonySnapshotMeta::VERSION;
+ meta.epoch = epoch;
+ meta.solutionCount = _solutionCount;
+ meta.recordSizeBytes = (unsigned int)sizeof(AntSolutionRecord);
+ meta.annPoolEntryBytes = (unsigned int)sizeof(PackedAnn);
+ meta.errorThreshold = _errorThreshold;
+ meta.initialTick = initialTick;
+ meta.anchorRingBytes = (unsigned int)sizeof(AnchorRing);
+ meta.exportSetBytes = (unsigned int)sizeof(ExportSet);
+ meta.rootSeed = _rootSeed;
+
+ // The meta, anchor ring and export set share one file. The file API writes a single contiguous
+ // buffer, so the three are gathered into the serial scratch: meta, then anchors, then export.
+ const unsigned long long headerBytes = sizeof(meta) + sizeof(AnchorRing) + sizeof(ExportSet);
+ static_assert(sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet) <= ANT_SNAPSHOT_SCRATCH_BYTES,
+ "ant snapshot header exceeds the scratch buffer");
+ unsigned char* headerBuffer = _snapshotScratch;
+ copyMem(headerBuffer, &meta, sizeof(meta));
+ copyMem(headerBuffer + sizeof(meta), _anchors, sizeof(AnchorRing));
+ copyMem(headerBuffer + sizeof(meta) + sizeof(AnchorRing), _exportSet, sizeof(ExportSet));
+ if (save(ANT_SNAPSHOT_HEADER_FILENAME, headerBytes, headerBuffer, directory) != (long long)headerBytes)
+ {
+ logToConsole(L"[ant-colony] failed to save snapshot header");
+ return false;
+ }
+
+ // Written even when the colony is empty, so the operator's snapshot is always the same number of files
+ const unsigned long long slots = antSnapshotSlotCount(_solutionCount);
+ const unsigned long long recordBytes = slots * sizeof(AntSolutionRecord);
+ const unsigned long long poolBytes = slots * sizeof(PackedAnn);
+ if (saveLargeFile(ANT_SNAPSHOT_RECORDS_FILENAME, recordBytes, (unsigned char*)_records, directory, false)
+ != (long long)recordBytes)
+ {
+ logToConsole(L"[ant-colony] failed to save snapshot records");
+ return false;
+ }
+ if (saveLargeFile(ANT_SNAPSHOT_POOL_FILENAME, poolBytes, (unsigned char*)_annPool, directory, false)
+ != (long long)poolBytes)
+ {
+ logToConsole(L"[ant-colony] failed to save snapshot pool");
+ return false;
+ }
+ return true;
+}
+
+template
+inline bool AntColony::loadSnapshot(unsigned short epoch, CHAR16* directory,
+ const m256i& rootSeed, unsigned int errorThreshold, unsigned int initialTick)
+{
+ ASSERT(_records != nullptr);
+ ASSERT(_annPool != nullptr);
+ ASSERT(_anchors != nullptr);
+
+ antSnapshotNameForEpoch(epoch);
+
+ // The meta, anchor ring and export set share one file. Read it whole, validate the meta before
+ // any colony state is touched, then copy the two sections into place.
+ const unsigned long long headerBytes = sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet);
+ static_assert(sizeof(AntColonySnapshotMeta) + sizeof(AnchorRing) + sizeof(ExportSet) <= ANT_SNAPSHOT_SCRATCH_BYTES,
+ "ant snapshot header exceeds the scratch buffer");
+ unsigned char* headerBuffer = _snapshotScratch;
+ if (load(ANT_SNAPSHOT_HEADER_FILENAME, headerBytes, headerBuffer, directory) != (long long)headerBytes)
+ {
+ logToConsole(L"[ant-colony] failed to load snapshot header");
+ return false;
+ }
+
+ AntColonySnapshotMeta meta;
+ copyMem(&meta, headerBuffer, sizeof(meta));
+ if (meta.magic != AntColonySnapshotMeta::MAGIC || meta.version != AntColonySnapshotMeta::VERSION)
+ {
+ antSnapshotFailure(L"bad magic/version", meta.magic, meta.version);
+ return false;
+ }
+ if (meta.epoch != epoch)
+ {
+ antSnapshotFailure(L"epoch mismatch, file/expected", meta.epoch, epoch);
+ return false;
+ }
+ // A differently sized record or ANN would parse cleanly and produce a tree that is silently wrong.
+ if (meta.recordSizeBytes != sizeof(AntSolutionRecord) || meta.annPoolEntryBytes != sizeof(PackedAnn))
+ {
+ antSnapshotFailure(L"layout mismatch, record/ann", meta.recordSizeBytes, meta.annPoolEntryBytes);
+ return false;
+ }
+ // The two sections after the meta must be exactly the size this build lays them out at, or the
+ // copies below would read them at the wrong offset.
+ if (meta.anchorRingBytes != sizeof(AnchorRing) || meta.exportSetBytes != sizeof(ExportSet))
+ {
+ antSnapshotFailure(L"layout mismatch, anchors/export", meta.anchorRingBytes, meta.exportSetBytes);
+ return false;
+ }
+ if (meta.solutionCount > ANT_MAX_NODES_PER_EPOCH)
+ {
+ antSnapshotFailure(L"solutionCount exceeds the cap, count/cap", meta.solutionCount, ANT_MAX_NODES_PER_EPOCH);
+ return false;
+ }
+ // Cross-check against the node state restored alongside this file. A mismatch means the two
+ // snapshots are not from the same moment - loading the tree anyway would derive every root from
+ // a seed the rest of the network is not using.
+ if (!(meta.rootSeed == rootSeed))
+ {
+ antSnapshotFailure(L"root seed does not match the restored node state, record/0", 0, 0);
+ return false;
+ }
+ if (meta.errorThreshold != errorThreshold)
+ {
+ antSnapshotFailure(L"threshold does not match the node, file/node", meta.errorThreshold, errorThreshold);
+ return false;
+ }
+ // Records hold absolute ticks; slotOf() maps them against initialTick, so a snapshot taken at a
+ // different base would resolve parent references to the wrong records. Refuse it.
+ if (meta.initialTick != initialTick)
+ {
+ antSnapshotFailure(L"initial tick does not match the node, file/node", meta.initialTick, initialTick);
+ return false;
+ }
+
+ // Everything above only read the meta, so a refusal there leaves the colony untouched. From here
+ // on the state is being overwritten
+ reset();
+
+ copyMem(_anchors, headerBuffer + sizeof(meta), sizeof(AnchorRing));
+ copyMem(_exportSet, headerBuffer + sizeof(meta) + sizeof(AnchorRing), sizeof(ExportSet));
+
+ // Read unconditionally and at the same sizing the save used, so an incomplete copy is refused
+ // here rather than booting a node with an empty tree.
+ const unsigned long long slots = antSnapshotSlotCount(meta.solutionCount);
+ const unsigned long long recordBytes = slots * sizeof(AntSolutionRecord);
+ const unsigned long long poolBytes = slots * sizeof(PackedAnn);
+ if (loadLargeFile(ANT_SNAPSHOT_RECORDS_FILENAME, recordBytes, (unsigned char*)_records, directory)
+ != (long long)recordBytes)
+ {
+ logToConsole(L"[ant-colony] failed to load snapshot records");
+ reset();
+ return false;
+ }
+ if (loadLargeFile(ANT_SNAPSHOT_POOL_FILENAME, poolBytes, (unsigned char*)_annPool, directory)
+ != (long long)poolBytes)
+ {
+ logToConsole(L"[ant-colony] failed to load snapshot pool");
+ reset();
+ return false;
+ }
+ if (_exportSet->count > ANT_EXPORT_MAX_SOLUTIONS)
+ {
+ antSnapshotFailure(L"export count exceeds the cap, count/cap", _exportSet->count, ANT_EXPORT_MAX_SOLUTIONS);
+ reset();
+ return false;
+ }
+ for (unsigned int i = 0; i < _exportSet->count; i++)
+ {
+ const unsigned int slot = _exportSet->order[i];
+ if (slot >= _exportSet->count)
+ {
+ antSnapshotFailure(L"export order out of range, position/slot", i, slot);
+ reset();
+ return false;
+ }
+ for (unsigned int j = 0; j < i; j++)
+ {
+ if (_exportSet->order[j] == slot)
+ {
+ antSnapshotFailure(L"export order duplicate, position/slot", i, slot);
+ reset();
+ return false;
+ }
+ }
+ }
+
+ // The caller's values, which the checks above proved the file agrees with.
+ _rootSeed = rootSeed;
+ _errorThreshold = errorThreshold;
+ _solutionCount = meta.solutionCount;
+ _initialTick = initialTick;
+
+ // Rebuild the intermediate data
+ if (!rebuildDerivedState())
+ {
+ reset();
+ return false;
+ }
+ return true;
+}
+
+template
+inline bool AntColony::rebuildDerivedState()
+{
+ // Hoisted out of the loop: unpacking each stored network to verify its hash needs somewhere to
+ // put it, and this path runs once at boot.
+ Ann annBuffer;
+
+ for (unsigned int i = 0; i < _solutionCount; i++)
+ {
+ const AntSolutionRecord& rec = _records[i];
+
+ unsigned int selfSlot = 0;
+ if (!slotOf(rec.selfRef.tick, selfSlot))
+ {
+ antSnapshotFailure(L"tick out of range, record/tick", i, rec.selfRef.tick);
+ return false;
+ }
+ // commit() writes the record and its network at the same index, and findIndexBySolutionRef
+ // and annOfNonRoot both rely on it
+ // annStateSlot point to the ANN that must have similar index to the record
+ if (rec.annStateSlot != i)
+ {
+ antSnapshotFailure(L"annStateSlot is not the record index, record/slot", i, rec.annStateSlot);
+ return false;
+ }
+
+ // The freshness rule validateChild() applied at admission, re-checked here so a tampered
+ // snapshot cannot smuggle in a record that was never admissible. anchorTick seeds the score's
+ // RNG, so it is consensus-relevant. The record's own absolute tick is its publish tick.
+ const unsigned int publishTick = rec.selfRef.tick;
+ if (rec.anchorTick > publishTick
+ || publishTick - rec.anchorTick > ANT_PUBLISH_WINDOW_TICKS)
+ {
+ antSnapshotFailure(L"anchor tick outside the freshness window, record/anchorTick", i, rec.anchorTick);
+ return false;
+ }
+
+ // Rebuild the tick index
+ AntTickSlot& tslot = _tickIndex[selfSlot];
+ if (tslot.count == 0)
+ {
+ tslot.startIdx = i;
+ }
+ else if (tslot.startIdx + tslot.count != i)
+ {
+ antSnapshotFailure(L"record breaks tick contiguity, record/tick", i, rec.selfRef.tick);
+ return false;
+ }
+ tslot.count++;
+
+ // The parent must be ROOT or an EARLIER record. The tick index built so far covers only
+ // records before this one, so a forward or self reference fails to resolve - which is what
+ // rejects a cycle.
+ const AntSolutionRecord* parentRec = nullptr;
+ if (!rec.parentRef.isRoot())
+ {
+ const long long parentIdx = findIndexBySolutionRef(rec.parentRef);
+ if (parentIdx == ANT_INVALID_INDEX || (unsigned long long)parentIdx >= i)
+ {
+ antSnapshotFailure(L"parent not an earlier record, record/parentTick", i, rec.parentRef.tick);
+ return false;
+ }
+ parentRec = &_records[parentIdx];
+ if (!(parentRec->pubkey == rec.pubkey))
+ {
+ antSnapshotFailure(L"parent belongs to another identity, record", i, 0);
+ return false;
+ }
+ }
+ const unsigned int expectedDepth = (parentRec != nullptr) ? (parentRec->depth + 1) : 1;
+ if (rec.depth != expectedDepth)
+ {
+ antSnapshotFailure(L"depth does not match the parent, record/depth", i, rec.depth);
+ return false;
+ }
+
+ // The two score rules validateChild() enforced when this record was admitted. A corrupt
+ // score would otherwise set a wrong bar for its own children.
+ if (rec.score > _errorThreshold)
+ {
+ antSnapshotFailure(L"score above the epoch threshold, record/score", i, rec.score);
+ return false;
+ }
+ if (parentRec != nullptr && rec.score >= parentRec->score)
+ {
+ antSnapshotFailure(L"score does not beat the parent, record/score", i, rec.score);
+ return false;
+ }
+
+ // Re-derive the hash from the stored network
+ _annPool[i].unpack(annBuffer.lut);
+ unsigned int annHash;
+ KangarooTwelve(&annBuffer, sizeof(annBuffer), &annHash, sizeof(annHash));
+ if (annHash != rec.childAnnHash)
+ {
+ antSnapshotFailure(L"stored network does not match childAnnHash, record", i, 0);
+ return false;
+ }
+
+ const AntDedupKey key{ rec.pubkey, rec.nonce, rec.parentRef };
+ if (_dedup->contains(key))
+ {
+ antSnapshotFailure(L"duplicate solution, record", i, 0);
+ return false;
+ }
+ if (_dedup->add(key) == QPI::NULL_INDEX)
+ {
+ antSnapshotFailure(L"dedup set full, record", i, 0);
+ return false;
+ }
+
+ // Rebuild the _childHeadByMiner and _childHeadByParent
+ unsigned int prevHead = NO_SIBLING;
+ if (rec.parentRef.isRoot())
+ {
+ _childHeadByMiner->get(rec.pubkey, prevHead);
+ _records[i].nextSiblingIdx = prevHead;
+ if (_childHeadByMiner->set(rec.pubkey, i) == QPI::NULL_INDEX)
+ {
+ antSnapshotFailure(L"miner index full, record", i, 0);
+ return false;
+ }
+ }
+ else
+ {
+ _childHeadByParent->get(rec.parentRef, prevHead);
+ _records[i].nextSiblingIdx = prevHead;
+ _childHeadByParent->set(rec.parentRef, i);
+ }
+
+ // Restore the stats also
+ _stats.acceptedSolutions++;
+ if (rec.depth > _stats.treeDepthMax)
+ {
+ _stats.treeDepthMax = rec.depth;
+ }
+ }
+
+ _stats.treeSizeCurrent = _solutionCount;
+ return true;
+}
diff --git a/src/mining/ant_colony/ant_colony_bpp9000.h b/src/mining/ant_colony/ant_colony_bpp9000.h
new file mode 100644
index 000000000..da88d0ab5
--- /dev/null
+++ b/src/mining/ant_colony/ant_colony_bpp9000.h
@@ -0,0 +1,8 @@
+#pragma once
+
+#include "mining/ant_colony/ant_colony.h"
+#include "score.h"
+
+// Binds the colony to bpp9000. This is the only place a concrete scorer is named, which is why
+// ant_colony.h itself can stay free of score.h and everything it drags in.
+using AntColonyBpp9000T = AntColony;
diff --git a/src/mining/ant_colony/ant_pending_solutions.h b/src/mining/ant_colony/ant_pending_solutions.h
new file mode 100644
index 000000000..b3658cb15
--- /dev/null
+++ b/src/mining/ant_colony/ant_pending_solutions.h
@@ -0,0 +1,434 @@
+#pragma once
+
+#include "platform/m256.h"
+#include "platform/concurrency.h"
+#include "platform/memory.h"
+#include "mining/ant_colony/ant_colony.h"
+
+// Solutions waiting to be published as transactions signed by the node's own computors
+// A queue is needed because a broadcast can be lost with nothing reporting it. An entry stores the
+// tick its transaction was targeted at, if it is not on-chain by then, publish again.
+struct AntPendingSolution
+{
+ m256i computorPublicKey;
+ m256i nonce;
+ SolutionRef parentRef;
+ unsigned int anchorTick; // ABSOLUTE. Bounds how long this entry is worth publishing.
+ unsigned int score; // computed at receipt; the publisher uses it without re-scoring
+};
+static_assert(sizeof(AntPendingSolution) == 32 + 32 + 8 + 8, "AntPendingSolution unexpected padding");
+
+class AntPendingSolutions
+{
+public:
+ static constexpr unsigned int CAPACITY = 65536;
+ static_assert((CAPACITY & (CAPACITY - 1)) == 0, "CAPACITY must be a power of two");
+ static constexpr unsigned int NO_ENTRY = 0xFFFFFFFFU;
+
+ // publicationTick[] states. A positive value is the tick the transaction was targeted at, which
+ // is also the deadline to see it on-chain
+ static constexpr int NOT_SCHEDULED = 0;
+ static constexpr int RECORDED = -1; // observed on-chain; never publish again
+ static constexpr int OBSOLETE = -2; // can never land; stop occupying the retry slot
+
+ struct Stats
+ {
+ unsigned long long received;
+ unsigned long long droppedNonCanonical;
+ unsigned long long droppedBadAnchor; // anchor in the future, or aged out of the ring
+ unsigned long long droppedParentUnknown; // parentRef names a node this node does not hold
+ unsigned long long droppedUnscorable; // the scorer returned no usable value
+ unsigned long long droppedUnacceptable; // scored, but the colony would reject it now
+ unsigned long long droppedDuplicate;
+ unsigned long long droppedFull;
+ unsigned long long published;
+ unsigned long long recorded;
+ unsigned long long obsoleteParentGone;
+ unsigned long long obsoleteExpired;
+ unsigned long long obsoleteGateRejected;
+ unsigned long long claimMismatch;
+ };
+
+ bool init()
+ {
+ setMem(this, sizeof(*this), 0);
+ if (!allocPoolWithErrorLog(L"AntPendingSolutions::_entries",
+ CAPACITY * sizeof(AntPendingSolution), (void**)&_entries, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntPendingSolutions::_publicationTick",
+ CAPACITY * sizeof(int), (void**)&_publicationTick, __LINE__))
+ {
+ return false;
+ }
+ if (!allocPoolWithErrorLog(L"AntPendingSolutions::_index",
+ INDEX_CAPACITY * sizeof(unsigned int), (void**)&_index, __LINE__))
+ {
+ return false;
+ }
+ reset();
+ return true;
+ }
+
+ void deinit()
+ {
+ if (_index)
+ {
+ freePool(_index);
+ }
+ if (_publicationTick)
+ {
+ freePool(_publicationTick);
+ }
+ if (_entries)
+ {
+ freePool(_entries);
+ }
+ _index = nullptr;
+ _publicationTick = nullptr;
+ _entries = nullptr;
+ }
+
+ void reset()
+ {
+ ASSERT(_entries != nullptr);
+ LockGuard guard(_lock);
+ setMem(_entries, CAPACITY * sizeof(AntPendingSolution), 0);
+ setMem(_publicationTick, CAPACITY * sizeof(int), 0);
+ setMem(_index, INDEX_CAPACITY * sizeof(unsigned int), 0xFF);
+ _count = 0;
+ _nextFree = 0;
+ _touchedSlots = 0;
+ setMem(&_stats, sizeof(_stats), 0);
+ }
+
+ // Ingress only counts what it drops, the caller decides what is worth dropping, because the
+ // reasons live where the colony can be consulted.
+ void noteReceived()
+ {
+ LockGuard guard(_lock);
+ _stats.received++;
+ }
+ void noteDroppedNonCanonical()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedNonCanonical++;
+ }
+ void noteDroppedBadAnchor()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedBadAnchor++;
+ }
+ void noteDroppedParentUnknown()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedParentUnknown++;
+ }
+ void noteDroppedDuplicate()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedDuplicate++;
+ }
+ void noteDroppedUnscorable()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedUnscorable++;
+ }
+ void noteDroppedUnacceptable()
+ {
+ LockGuard guard(_lock);
+ _stats.droppedUnacceptable++;
+ }
+
+ void getStats(Stats& outStats, unsigned int& outCount) const
+ {
+ LockGuard guard(_lock);
+ outStats = _stats;
+ outCount = _count;
+ }
+
+ // Queue a solution for publication, called from request processors.
+ bool add(const m256i& computorPublicKey, const SolutionRef& parentRef,
+ unsigned int anchorTick, unsigned int score, const m256i& nonce)
+ {
+ LockGuard guard(_lock);
+ const unsigned int slot = indexSlotFor(computorPublicKey, parentRef, nonce);
+ unsigned int entryIdx = NO_ENTRY;
+ if (_index[slot] != INDEX_EMPTY)
+ {
+ // An OBSOLETE entry is not a duplicate
+ // Every other state is a real duplicate: NOT_SCHEDULED and scheduled are still live, and
+ // RECORDED already landed, so a resend would be rejected as a replay anyway.
+ if (_publicationTick[_index[slot]] != OBSOLETE)
+ {
+ _stats.droppedDuplicate++;
+ return false;
+ }
+ entryIdx = _index[slot];
+ }
+ else
+ {
+ entryIdx = findFreeEntry();
+ if (entryIdx == NO_ENTRY)
+ {
+ _stats.droppedFull++;
+ return false;
+ }
+ }
+
+ AntPendingSolution& e = _entries[entryIdx];
+ e.computorPublicKey = computorPublicKey;
+ e.nonce = nonce;
+ e.parentRef = parentRef;
+ e.anchorTick = anchorTick;
+ e.score = score;
+ _publicationTick[entryIdx] = NOT_SCHEDULED;
+ if (_index[slot] == INDEX_EMPTY)
+ {
+ _index[slot] = entryIdx;
+ _count++;
+ }
+ return true;
+ }
+
+ // Pick the next solution this computor should publish, or NO_ENTRY.
+ // Anything already past its publish window is retired here rather than published
+ unsigned int selectForPublish(const m256i& computorPublicKey, unsigned int currentTick,
+ AntPendingSolution& outEntry)
+ {
+ LockGuard guard(_lock);
+
+ unsigned int found = scanForPublish(computorPublicKey, currentTick, true);
+ if (found == NO_ENTRY)
+ {
+ found = scanForPublish(computorPublicKey, currentTick, false);
+ }
+ if (found != NO_ENTRY)
+ {
+ outEntry = _entries[found];
+ }
+ return found;
+ }
+
+ void markScheduled(unsigned int index, int publicationTick)
+ {
+ LockGuard guard(_lock);
+ if (index >= CAPACITY || _publicationTick[index] < 0)
+ {
+ return;
+ }
+ _publicationTick[index] = publicationTick;
+ _stats.published++;
+ }
+
+ void markObsoleteParentGone(unsigned int index)
+ {
+ retire(index, _stats.obsoleteParentGone);
+ }
+ void markObsoleteExpired(unsigned int index)
+ {
+ retire(index, _stats.obsoleteExpired);
+ }
+ void markObsoleteGateRejected(unsigned int index)
+ {
+ retire(index, _stats.obsoleteGateRejected);
+ }
+
+ void noteClaimMismatch()
+ {
+ LockGuard guard(_lock);
+ _stats.claimMismatch++;
+ }
+
+ void markRecorded(const m256i& computorPublicKey, const SolutionRef& parentRef, const m256i& nonce)
+ {
+ LockGuard guard(_lock);
+
+ const unsigned int slot = indexSlotFor(computorPublicKey, parentRef, nonce);
+ if (_index[slot] != INDEX_EMPTY)
+ {
+ _publicationTick[_index[slot]] = RECORDED;
+ _stats.recorded++;
+ return;
+ }
+
+ const unsigned int entryIdx = findFreeEntry();
+ if (entryIdx == NO_ENTRY)
+ {
+ // Nothing to reclaim
+ return;
+ }
+
+ AntPendingSolution& e = _entries[entryIdx];
+ e.computorPublicKey = computorPublicKey;
+ e.nonce = nonce;
+ e.parentRef = parentRef;
+ e.anchorTick = 0;
+ e.score = 0;
+ _publicationTick[entryIdx] = RECORDED;
+ _index[slot] = entryIdx;
+ _count++;
+ _stats.recorded++;
+ }
+
+private:
+ static constexpr unsigned int INDEX_CAPACITY = 2 * CAPACITY;
+ // Not INDEX_EMPTY: network_messages/assets.h defines that as a macro, and this header is included
+ // after it in qubic.cpp.
+ static constexpr unsigned int INDEX_EMPTY = 0xFFFFFFFFU;
+
+ // A slot is free if it has never been used or if its entry is finished. Reuse is IN PLACE:
+ // nothing is ever moved, so an index the tick processor is holding across the publish gate
+ // stays valid.
+ unsigned int findFreeEntry()
+ {
+ for (unsigned int i = 0; i < CAPACITY; i++)
+ {
+ const unsigned int idx = (_nextFree + i) & (CAPACITY - 1);
+ if (isZero(_entries[idx].computorPublicKey))
+ {
+ claim(idx);
+ return idx;
+ }
+ if (_publicationTick[idx] == RECORDED || _publicationTick[idx] == OBSOLETE)
+ {
+ indexRemove(_entries[idx]);
+ _count--;
+ claim(idx);
+ return idx;
+ }
+ }
+ return NO_ENTRY;
+ }
+
+ void claim(unsigned int idx)
+ {
+ _nextFree = (idx + 1) & (CAPACITY - 1);
+ if (idx + 1 > _touchedSlots)
+ {
+ _touchedSlots = idx + 1;
+ }
+ }
+
+ unsigned int scanForPublish(const m256i& computorPublicKey, unsigned int currentTick, bool retries)
+ {
+ for (unsigned int i = 0; i < _touchedSlots; i++)
+ {
+ const int state = _publicationTick[i];
+ if (retries)
+ {
+ if (state <= NOT_SCHEDULED || state > (int)currentTick)
+ {
+ continue;
+ }
+ }
+ else if (state != NOT_SCHEDULED)
+ {
+ continue;
+ }
+ if (isZero(_entries[i].computorPublicKey) || !(_entries[i].computorPublicKey == computorPublicKey))
+ {
+ continue;
+ }
+ if (currentTick - _entries[i].anchorTick > ANT_PUBLISH_WINDOW_TICKS)
+ {
+ retireLocked(i, _stats.obsoleteExpired);
+ continue;
+ }
+ return i;
+ }
+ return NO_ENTRY;
+ }
+
+ void retire(unsigned int index, unsigned long long& counter)
+ {
+ LockGuard guard(_lock);
+ retireLocked(index, counter);
+ }
+
+ void retireLocked(unsigned int index, unsigned long long& counter)
+ {
+ if (index >= CAPACITY || _publicationTick[index] < 0)
+ {
+ return;
+ }
+ _publicationTick[index] = OBSOLETE;
+ counter++;
+ }
+
+ static AntDedupKey keyOf(const m256i& computorPublicKey, const SolutionRef& parentRef,
+ const m256i& nonce)
+ {
+ AntDedupKey key;
+ key.pubkey = computorPublicKey;
+ key.nonce = nonce;
+ key.parentRef = parentRef;
+ return key;
+ }
+
+ // Reads only the low words would put every nonce differing above them in one slot, which is the
+ // clustering the index exists to avoid.
+ static unsigned int hashOf(const m256i& computorPublicKey, const SolutionRef& parentRef,
+ const m256i& nonce)
+ {
+ const AntDedupKey key = keyOf(computorPublicKey, parentRef, nonce);
+ unsigned long long digest;
+ KangarooTwelve(&key, sizeof(key), &digest, sizeof(digest));
+ return (unsigned int)(digest & (INDEX_CAPACITY - 1));
+ }
+
+ // Returns the slot holding this key, or the first free slot if it is absent. Linear probing over
+ // a table at most half full, so the walk always terminates.
+ unsigned int indexSlotFor(const m256i& computorPublicKey, const SolutionRef& parentRef,
+ const m256i& nonce) const
+ {
+ unsigned int slot = hashOf(computorPublicKey, parentRef, nonce);
+ for (unsigned int probe = 0; probe < INDEX_CAPACITY; probe++)
+ {
+ const unsigned int e = _index[slot];
+ if (e == INDEX_EMPTY)
+ {
+ return slot;
+ }
+ if (keyOf(_entries[e].computorPublicKey, _entries[e].parentRef, _entries[e].nonce)
+ == keyOf(computorPublicKey, parentRef, nonce))
+ {
+ return slot;
+ }
+ slot = (slot + 1) & (INDEX_CAPACITY - 1);
+ }
+ return 0;
+ }
+
+ void indexRemove(const AntPendingSolution& entry)
+ {
+ const unsigned int slot = indexSlotFor(entry.computorPublicKey, entry.parentRef, entry.nonce);
+ if (_index[slot] == INDEX_EMPTY)
+ {
+ return;
+ }
+ _index[slot] = INDEX_EMPTY;
+
+ // Re-place the run that followed it, or a probe would stop early at the hole just made.
+ unsigned int next = (slot + 1) & (INDEX_CAPACITY - 1);
+ while (_index[next] != INDEX_EMPTY)
+ {
+ const unsigned int moved = _index[next];
+ _index[next] = INDEX_EMPTY;
+ const unsigned int target = indexSlotFor(_entries[moved].computorPublicKey,
+ _entries[moved].parentRef, _entries[moved].nonce);
+ _index[target] = moved;
+ next = (next + 1) & (INDEX_CAPACITY - 1);
+ }
+ }
+
+ AntPendingSolution* _entries;
+ int* _publicationTick;
+ unsigned int* _index;
+ unsigned int _count;
+ unsigned int _nextFree;
+ unsigned int _touchedSlots;
+ Stats _stats;
+ mutable volatile char _lock;
+};
diff --git a/src/mining/mining.h b/src/mining/mining.h
index c4796371f..f4dd84bae 100644
--- a/src/mining/mining.h
+++ b/src/mining/mining.h
@@ -10,6 +10,11 @@
#include
+// Miners tracked in the ranking table that feeds computor selection. A hard cap rather than mere
+// sizing: once full, a newcomer is admitted only if it outranks the current worst entry. Lives here
+// rather than in qubic.cpp so mining headers can size per-miner structures against it.
+#define MAX_NUMBER_OF_MINERS 8192
+
static unsigned int getTickInDogeBroadcastCycle()
{
#ifdef NO_UEFI
@@ -353,3 +358,41 @@ struct CustomMiningStats
};
static CustomMiningStats gDogeMiningStats;
+
+// Ant colony solution transaction
+constexpr int ANT_COLONY_MINING_SOLUTION_INPUT_TYPE = 12;
+struct AntColonyMiningSolutionTransaction : public Transaction
+{
+ static constexpr unsigned char transactionType()
+ {
+ return ANT_COLONY_MINING_SOLUTION_INPUT_TYPE;
+ }
+
+ static constexpr long long minAmount()
+ {
+ return SOLUTION_SECURITY_DEPOSIT; // same anti-spam deposit as legacy
+ }
+
+ static constexpr unsigned short minInputSize()
+ {
+ return sizeof(parentTick) + sizeof(parentSolutionIndexInTick) + sizeof(anchorTick) + sizeof(claimedScore) + sizeof(nonce); // 4 + 4 + 4 + 4 + 32 = 48 bytes
+ }
+
+ static bool isSolutionTransaction(const Transaction* tx)
+ {
+ return isZero(tx->destinationPublicKey)
+ && tx->inputType == transactionType()
+ && tx->amount >= minAmount()
+ && tx->inputSize == minInputSize();
+ }
+
+ unsigned int parentTick; // ABSOLUTE tick of the parent ref
+ unsigned int parentSolutionIndexInTick; // dense within-tick index of the parent ref
+ unsigned int anchorTick; // ABSOLUTE tick whose digest the solution anchored to (RNG seed + freshness)
+ // The score the submitter claims this solution reaches. The deposit is refunded only when it
+ // matches the score the node computes
+ unsigned int claimedScore;
+ m256i nonce;
+ unsigned char signature[SIGNATURE_SIZE];
+};
+static_assert(sizeof(AntColonyMiningSolutionTransaction) == sizeof(Transaction) + 4 + 4 + 4 + 4 + 32 + SIGNATURE_SIZE, "AntColonyMiningSolutionTransaction unexpected padding");
diff --git a/src/mining/score_bpp9000.h b/src/mining/score_bpp9000.h
index b7f308b40..40468c8c4 100644
--- a/src/mining/score_bpp9000.h
+++ b/src/mining/score_bpp9000.h
@@ -45,6 +45,15 @@ struct ScoreBpp9000
static_assert(lutSize <= lutStride, "LUT rows must fit the padded stride");
+ // K is a real degree of freedom here: the walk restores K = nonce[2] as its explore-step count
+ static bool isCanonicalAntNonce(const unsigned char* nonce)
+ {
+ return (getAlgoType(nonce) == AlgoType::Bpp9000)
+ && (nonce[1] >= 1)
+ && (nonce[1] <= MAX_LUT_ENTRIES_PER_STEP)
+ && (nonce[2] <= numberOfMutations);
+ }
+
// random2 draw sizes padded up to a multiple of 64 bytes; leading bytes bit-exact with reference.
static constexpr unsigned long long lutInitBytes = maxNumberOfNeurons * lutSize;
static constexpr unsigned long long lutInitPaddedBytes = ((lutInitBytes + 63) / 64) * 64;
@@ -79,13 +88,26 @@ struct ScoreBpp9000
kEvolution,
};
- // Rollback/snapshot state: just the per-neuron LUT.
+ // An ANN is its per-neuron LUT: maxNumberOfNeurons rows of lutSize entries
+ // resourceTestingDigest, written to snapshots, sent to miners...
struct ANN
+ {
+ unsigned char lut[maxNumberOfNeurons * lutSize];
+ };
+
+ // padding LUT for a SIMD
+ struct PaddedLut
{
alignas(64) unsigned char lut[maxNumberOfNeurons * lutStride];
};
- ANN currentANN;
- ANN prevANN;
+
+ PaddedLut currentANN;
+ PaddedLut prevANN;
+
+ // LUT that produced the score returned by the walk, in working layout. The ant colony stores this
+ // as the child's inherited state, so a child branches from the best-ever LUT rather than the last
+ // one walked; read it with getBestANN().
+ PaddedLut bestANN;
struct InitValue
{
@@ -403,7 +425,8 @@ struct ScoreBpp9000
#endif
}
- // Sliding-window self-clocked score via the window-batched SIMD kernel (AVX-512 or AVX2, bit-exact).
+ // Sliding-window self-clocked score via the window-batched SIMD kernel
+ // Apply on the curANN
unsigned int score()
{
// PROFILE_NAMED_SCOPE("bpp9000:score");
@@ -972,44 +995,102 @@ struct ScoreBpp9000
currentANN.lut[storageIdx] = newTrit;
}
- // Seed the ANN: root LUT from the pubkey alone (each computor's fixed root); mutation seeds from
- // pubkey+nonce (nonce[0..2] are the algo/L/K knobs, excluded from the RNG). Returns the start score.
- unsigned int initializeANN(const unsigned char* publicKey, const unsigned char* nonce, const unsigned char* pRandom2Pool)
+ // Derive the root LUT material
+ void deriveRootLut(const unsigned char* publicKey, const unsigned char* pRandom2Pool)
{
- // PROFILE_NAMED_SCOPE("bpp9000:initializeANN");
unsigned char rootHash[32];
KangarooTwelve(publicKey, 32, rootHash, 32);
random2(rootHash, pRandom2Pool, (unsigned char*)&initValue.lutInit, lutInitPaddedBytes);
+ }
+ // Derive the mutation-walk seeds. nonce[0..2] are the algo/L/K knobs and stay excluded from the RNG,
+ // so K and L can be chosen freely without reseeding the walk. anchorTickDigest == nullptr for the
+ // standalone walk; the ant colony binds a child's walk to the tick it anchors on.
+ void deriveMutationSeeds(
+ const unsigned char* publicKey,
+ const unsigned char* nonce,
+ const unsigned char* anchorTickDigest,
+ const unsigned char* pRandom2Pool)
+ {
unsigned char searchHash[32];
- unsigned char combined[64];
+ unsigned char combined[96];
copyMem(combined, publicKey, 32);
copyMem(combined + 32, nonce, 32);
combined[32] = 0;
combined[33] = 0;
combined[34] = 0;
- KangarooTwelve(combined, 64, searchHash, 32);
+ unsigned int combinedSize = 64;
+ if (anchorTickDigest != nullptr)
+ {
+ copyMem(combined + 64, anchorTickDigest, 32);
+ combinedSize = 96;
+ }
+ KangarooTwelve(combined, combinedSize, searchHash, 32);
random2(searchHash, pRandom2Pool, (unsigned char*)&initValue.mutationSeed, mutationSeedPaddedBytes);
+ }
+
+ // Store the LUT densely by updated-neuron position k (row k): row k holds neuron
+ // updatedNeuronIndices[k]'s LUT. RNG draw into initValue.lutInit unchanged (bit-exact).
+ void applyRootLut(PaddedLut& target)
+ {
+ // The loop below writes only rows [0, numberOfUpdatedNeurons) columns [0, lutSize), and the
+ // SIMD path loads whole rows, so the rest has to be initialised rather than left as whatever
+ // the buffer previously held.
+ setMem(&target, sizeof(target), 0);
- // Store the LUT densely by updated-neuron position k (row k): row k holds neuron
- // updatedNeuronIndices[k]'s LUT. RNG draw into initValue.lutInit unchanged (bit-exact).
for (unsigned long long k = 0; k < numberOfUpdatedNeurons; ++k)
{
const unsigned long long n = updatedNeuronIndices[k];
for (unsigned long long line = 0; line < lutSize; ++line)
{
- currentANN.lut[k * lutStride + line] = (unsigned char)(initValue.lutInit[n * lutSize + line] % 3);
+ target.lut[k * lutStride + line] = (unsigned char)(initValue.lutInit[n * lutSize + line] % 3);
}
}
+ }
+
+ // Working layout to ANN remove the stride padding.
+ void compact(const PaddedLut& src, ANN& out) const
+ {
+ for (unsigned long long k = 0; k < maxNumberOfNeurons; ++k)
+ {
+ copyMem(out.lut + k * lutSize, src.lut + k * lutStride, lutSize);
+ }
+ }
+
+ // Restores the stride and zeroes the padding.
+ void expand(const ANN& src, PaddedLut& out) const
+ {
+ setMem(&out, sizeof(out), 0);
+ for (unsigned long long k = 0; k < maxNumberOfNeurons; ++k)
+ {
+ copyMem(out.lut + k * lutStride, src.lut + k * lutSize, lutSize);
+ }
+ }
+
+ // The LUT behind the score the last walk returned.
+ void getBestANN(ANN& out) const
+ {
+ compact(bestANN, out);
+ }
+
+ // Seed the ANN: root LUT from the pubkey alone (each computor's fixed root); mutation seeds from
+ // pubkey+nonce (nonce[0..2] are the algo/L/K knobs, excluded from the RNG). Returns the start score.
+ unsigned int initializeANN(
+ const unsigned char* publicKey,
+ const unsigned char* nonce,
+ const unsigned char* pRandom2Pool)
+ {
+ // PROFILE_NAMED_SCOPE("bpp9000:initializeANN");
+ deriveRootLut(publicKey, pRandom2Pool);
+ deriveMutationSeeds(publicKey, nonce, nullptr, pRandom2Pool);
+ applyRootLut(currentANN);
return score();
}
- // Anti-attractor search: L mutations/step; accept worse-or-equal for the first K steps (explore),
- // then better-or-equal (exploit); one-step rollback; keep and return the best score found.
- unsigned int computeScore(const unsigned char* publicKey, const unsigned char* nonce, const unsigned char* pRandom2Pool)
+ // Miner-chosen number of LUT entries rewritten per step, clamped to the verifiable range.
+ static unsigned int lutEntriesPerStep(const unsigned char* nonce)
{
- // PROFILE_NAMED_SCOPE("bpp9000:computeScore");
unsigned int L = nonce[1];
if (L < 1)
{
@@ -1019,11 +1100,17 @@ struct ScoreBpp9000
{
L = MAX_LUT_ENTRIES_PER_STEP;
}
- // Explore disabled pre-ant-colony (K=0); restore K = nonce[2] when ants return.
- const unsigned long long K = 0;
+ return L;
+ }
- unsigned int cur = initializeANN(publicKey, nonce, pRandom2Pool);
+ // Anti-attractor walk starting from the LUT already in currentANN: L mutations/step; accept
+ // worse-or-equal for the first K steps (explore), then better-or-equal (exploit); one-step rollback.
+ // Returns the best score found and leaves the LUT that produced it in bestANN.
+ unsigned int computeScoreFromCurrent(unsigned int L, unsigned long long K, unsigned int startScore)
+ {
+ unsigned int cur = startScore;
unsigned int best = cur;
+ copyMem(&bestANN, ¤tANN, sizeof(bestANN));
for (unsigned long long s = 0; s < numberOfMutations; ++s)
{
@@ -1058,11 +1145,68 @@ struct ScoreBpp9000
if (cur < best)
{
best = cur;
+ copyMem(&bestANN, ¤tANN, sizeof(bestANN));
}
}
return best;
}
+ // Anti-attractor search: L mutations/step; accept worse-or-equal for the first K steps (explore),
+ // then better-or-equal (exploit); one-step rollback; keep and return the best score found.
+ unsigned int computeScore(
+ const unsigned char* publicKey,
+ const unsigned char* nonce,
+ const unsigned char* pRandom2Pool)
+ {
+ // PROFILE_NAMED_SCOPE("bpp9000:computeScore");
+ const unsigned int L = lutEntriesPerStep(nonce);
+ // Explore disabled for the standalone algorithm (K=0); the ant colony passes K = nonce[2].
+ const unsigned long long K = 0;
+
+ const unsigned int cur = initializeANN(publicKey, nonce, pRandom2Pool);
+
+ return computeScoreFromCurrent(L, K, cur);
+ }
+
+ // Ant colony: the network every one of an identity's lineages starts from. Written to a buffer the
+ // caller owns, so two roots can be derived on one engine without the first silently becoming the
+ // second - a child scored against the wrong root would differ only in resourceTestingDigest.
+ // Uses currentANN as its working buffer, so it destroys whatever the engine was holding. Callers
+ // derive a root and then score from it, which overwrites currentANN anyway.
+ void deriveRootANN(const unsigned char* publicKey, const unsigned char* pRandom2Pool, ANN& out)
+ {
+ deriveRootLut(publicKey, pRandom2Pool);
+ applyRootLut(currentANN);
+ compact(currentANN, out);
+ }
+
+ // Ant colony: score a child by inheriting the parent's LUT and walking it with the child's own seeds
+ unsigned int computeScoreFromParent(
+ const ANN& parentANN,
+ const unsigned char* publicKey,
+ const unsigned char* nonce,
+ const unsigned char* anchorTickDigest,
+ const unsigned char* pRandom2Pool)
+ {
+ // The canonical rule
+ if (!isCanonicalAntNonce(nonce))
+ {
+ return INVALID_SCORE_VALUE;
+ }
+
+ // Get the ANN from parent, also init the new mutation starting point
+ expand(parentANN, currentANN);
+ deriveMutationSeeds(publicKey, nonce, anchorTickDigest, pRandom2Pool);
+
+ // Both knobs are already in range: the check above is what puts them there.
+ const unsigned int L = lutEntriesPerStep(nonce);
+ const unsigned long long K = nonce[2];
+
+ const unsigned int cur = score();
+
+ return computeScoreFromCurrent(L, K, cur);
+ }
+
int getLastOutput(unsigned char* requestedOutput, int requestedSizeInBytes)
{
return 0;
diff --git a/src/mining/score_engine.h b/src/mining/score_engine.h
index d6752a80e..a95cb0482 100644
--- a/src/mining/score_engine.h
+++ b/src/mining/score_engine.h
@@ -61,6 +61,20 @@ struct ScoreEngine
}
}
+ // Each engine owns its canonical ant-nonce rule; this switch is the algorithm seam, so ingress
+ // code stays algorithm-agnostic. Neuraxon is reserved and not ant-minable, so no nonce in its
+ // slot is canonical.
+ static bool isCanonicalAntNonce(const unsigned char* nonce)
+ {
+ switch (getAlgoType(nonce))
+ {
+ case AlgoType::Bpp9000:
+ return ScoreBpp9000::isCanonicalAntNonce(nonce);
+ default:
+ return false;
+ }
+ }
+
// returns last computed output neurons of the active bpp9000 slot
m256i getLastOutput()
{
diff --git a/src/mining/trit_pack.h b/src/mining/trit_pack.h
new file mode 100644
index 000000000..bd3d86ef4
--- /dev/null
+++ b/src/mining/trit_pack.h
@@ -0,0 +1,51 @@
+#pragma once
+
+// Ternary storage: values {0,1,2} at 2 bits each.
+//
+// Qubic's mining networks store their genome as trits
+// One byte per trit wastes six bits of eight; two bits per trit cuts the stored genome to a quarter.
+// Callers hash and transmit the unpacked bytes
+
+namespace score_engine
+{
+
+template
+struct PackedTrits
+{
+ static_assert(GROUPS > 0, "need at least one group");
+ static_assert(TRITS_PER_GROUP > 0, "a group needs at least one trit");
+ static_assert(TRITS_PER_GROUP * 2 <= 64, "a group must fit at 2 bits per trit in one uint64");
+
+ static constexpr unsigned long long groupCount = GROUPS;
+ static constexpr unsigned long long tritsPerGroup = TRITS_PER_GROUP;
+ static constexpr unsigned long long tritCount = GROUPS * TRITS_PER_GROUP;
+
+ unsigned long long word[GROUPS];
+
+ void pack(const unsigned char* src)
+ {
+ for (unsigned long long g = 0; g < GROUPS; g++)
+ {
+ unsigned long long packed = 0;
+ for (unsigned long long i = 0; i < TRITS_PER_GROUP; i++)
+ {
+ packed |= ((unsigned long long)(src[g * TRITS_PER_GROUP + i] & 3u)) << (i * 2);
+ }
+ word[g] = packed;
+ }
+ }
+
+ void unpack(unsigned char* dst) const
+ {
+ for (unsigned long long g = 0; g < GROUPS; g++)
+ {
+ const unsigned long long packed = word[g];
+ for (unsigned long long i = 0; i < TRITS_PER_GROUP; i++)
+ {
+ dst[g * TRITS_PER_GROUP + i] = (unsigned char)((packed >> (i * 2)) & 3ull);
+ }
+ }
+ }
+};
+
+}
diff --git a/src/network_messages/all.h b/src/network_messages/all.h
index edd8e66ef..214242922 100644
--- a/src/network_messages/all.h
+++ b/src/network_messages/all.h
@@ -17,3 +17,4 @@
#include "transactions.h"
#include "system_info.h"
#include "revenue_data.h"
+#include "ant_colony_message.h"
diff --git a/src/network_messages/ant_colony_message.h b/src/network_messages/ant_colony_message.h
new file mode 100644
index 000000000..98b123eb8
--- /dev/null
+++ b/src/network_messages/ant_colony_message.h
@@ -0,0 +1,177 @@
+#pragma once
+
+#include "common_def.h"
+
+// Asks for the parents one identity can branch a child from. Scoped by pubkey because a child must
+// name a parent in its OWN tree - validate() rejects anything else with RejectWrongTree -
+// Operator-signed: the request payload is followed by SIGNATURE_SIZE bytes signed by
+// operatorPublicKey. Signature only, with no monotonic nonce. A nonce exists to make an operator
+// ACTION execute exactly once; replaying a read just costs a duplicate answer, while consuming the
+// nonce would put a polling miner in contention with every other operator command.
+//
+// Paginated via fromIndex / nextIndex.
+struct RequestAntIdentityTree
+{
+ // Whose tree to report. Usually the caller's own.
+ m256i pubkey;
+ // Record index to resume scanning from (0 on the first call).
+ unsigned int fromIndex;
+ unsigned int padding;
+ static constexpr unsigned char type()
+ {
+ return REQUEST_ANT_IDENTITY_TREE;
+ }
+};
+static_assert(sizeof(RequestAntIdentityTree) == 40, "RequestAntIdentityTree unexpected size");
+
+// A pool miner hands its computor a solution over BroadcastMessage(MESSAGE_TYPE_ANT_SOLUTION); this
+// is the payload that follows the header.
+struct AntSolutionBroadcastPayload
+{
+ unsigned int parentTick; // ABSOLUTE
+ unsigned int parentSolutionIndexInTick;
+ unsigned int anchorTick; // ABSOLUTE
+ unsigned int claimedScore;
+ m256i nonce;
+};
+static_assert(sizeof(AntSolutionBroadcastPayload) == 48, "AntSolutionBroadcastPayload unexpected size");
+
+// Max identity-tree nodes returned per response. Miners page through the
+// rest via the nextIndex cursor.
+constexpr unsigned int ANT_IDENTITY_TREE_NODES_PER_RESPONSE = 64;
+
+// Max records scanned per request
+constexpr unsigned int ANT_IDENTITY_TREE_SCAN_BUDGET = 1024;
+
+// One stored node of the requested identity's tree. selfTick/selfSolutionIndexInTick is the
+// ref a child sets as its own parentRef to extend this node; parentTick/parentSolutionIndexInTick
+// is this node's OWN parent - (0, 0xFFFFFFFF) means the root - so paging every node of a pubkey
+// reconstructs the whole tree, edges included, without fetching any network bytes.
+// The score is an error count, so smaller is better: a child must score strictly below score.
+// childCount is how many children this node already holds, capped at ANT_MAX_CHILDREN_PER_PARENT; at
+// the cap it takes no more children (0 for the cap means unbound).
+struct AntIdentityTreeNode
+{
+ unsigned int selfTick;
+ unsigned int selfSolutionIndexInTick;
+ unsigned int parentTick;
+ unsigned int parentSolutionIndexInTick;
+ unsigned int score;
+ unsigned int childCount;
+ unsigned int anchorTick; // this node's own anchor tick number (ABSOLUTE)
+ unsigned int depth;
+};
+static_assert(sizeof(AntIdentityTreeNode) == 32, "AntIdentityTreeNode unexpected size");
+
+// Metadata header only; followed by count * AntIdentityTreeNode (count * itemSize
+// bytes). itemSize lets the receiver validate the payload without hardcoding the
+// entry size.
+struct RespondAntIdentityTreeHeader
+{
+ // Number of AntIdentityTreeNode entries that follow this header.
+ unsigned int count;
+ // Size in bytes of one AntIdentityTreeNode entry.
+ unsigned int itemSize;
+ // Resume cursor for the next request; 0 means no more records.
+ unsigned int nextIndex;
+ static constexpr unsigned char type()
+ {
+ return RESPOND_ANT_IDENTITY_TREE;
+ }
+};
+static_assert(sizeof(RespondAntIdentityTreeHeader) == 12, "RespondAntIdentityTreeHeader unexpected size");
+
+// The largest an identity-tree response can be, the header followed by a full page of entries
+struct AntIdentityTreeResponse
+{
+ RespondAntIdentityTreeHeader header;
+ AntIdentityTreeNode items[ANT_IDENTITY_TREE_NODES_PER_RESPONSE];
+};
+static_assert(sizeof(AntIdentityTreeResponse)
+ == sizeof(RespondAntIdentityTreeHeader)
+ + ANT_IDENTITY_TREE_NODES_PER_RESPONSE * sizeof(AntIdentityTreeNode),
+ "AntIdentityTreeResponse must have no padding between the header and the items");
+
+// RespondAntParentAnnHeader.status values.
+constexpr unsigned char ANT_PARENT_ANN_STATUS_OK = 0; // ANN bytes follow the header
+constexpr unsigned char ANT_PARENT_ANN_STATUS_NOT_FOUND = 1; // parentRef has no record
+constexpr unsigned char ANT_PARENT_ANN_STATUS_IS_ROOT = 2; // ROOT_REF; no ANN payload - miner derives its own per-identity root
+
+// ONE tree node's stored network, named by parentRef - the ANN state a miner mutates to extend
+// that node. The tree itself is listed by the identity-tree query; this fetches the material for a
+// single chosen parent.
+// Operator-signed: the request payload is followed by SIGNATURE_SIZE bytes signed by
+// operatorPublicKey
+struct RequestAntParentAnn
+{
+ unsigned int parentRefTick;
+ unsigned int parentRefSolutionIndexInTick;
+ static constexpr unsigned char type()
+ {
+ return REQUEST_ANT_PARENT_ANN;
+ }
+};
+static_assert(sizeof(RequestAntParentAnn) == 8, "RequestAntParentAnn unexpected size");
+
+// Metadata header, when status is Ok, annSizeBytes bytes of CANONICAL ANN follow it - one trit per
+// byte, the form the scorer consumes, so the receiver does no unpacking. annSizeBytes is 0 for every
+// other status. Kept ANN-agnostic here to avoid a heavy include; the receiver reads the trailing
+// blob by annSizeBytes.
+struct RespondAntParentAnnHeader
+{
+ unsigned int parentRefTick;
+ unsigned int parentRefSolutionIndexInTick;
+ // Bytes of canonical ANN that follow this header: ANN LUT size when status is Ok, 0 for every other
+ // status.
+ unsigned int annSizeBytes;
+ unsigned char status;
+ unsigned char padding[3];
+ static constexpr unsigned char type()
+ {
+ return RESPOND_ANT_PARENT_ANN;
+ }
+};
+static_assert(sizeof(RespondAntParentAnnHeader) == 16, "RespondAntParentAnnHeader unexpected size");
+
+struct RequestAntEpochContext
+{
+ static constexpr unsigned char type()
+ {
+ return REQUEST_ANT_EPOCH_CONTEXT;
+ }
+};
+
+// Per-epoch ant-colony parameters a miner needs to start building solutions:
+// the score threshold, the freshness window, the epoch's root seed, pool occupancy,
+// and the per-parent child cap.
+// The anchor digest is not included; a miner derives it from the anchor tick's TickData
+// (REQUEST_TICK_DATA): transactionDigest = K12(TickData), then K12(anchorTick || transactionDigest).
+#pragma pack(push, 1)
+struct RespondAntEpochContext
+{
+ // The epoch-start spectrum digest
+ m256i spectrumDigest;
+ // confirm its task file matches the one the node scores against.
+ m256i topologyHash;
+ m256i dataHash;
+ // score threshold for this epoch
+ unsigned int threshold;
+ // ANT_PUBLISH_WINDOW_TICKS: publish within this many ticks of the anchor.
+ unsigned int freshnessWindow;
+ // accepted solutions so far this epoch
+ unsigned int solutionCount;
+ // free slots in the live ANN pool
+ unsigned int freeAnnSlotsCount;
+ // ANT_MAX_CHILDREN_PER_PARENT: max children a parent takes; 0 = unbound
+ unsigned int maxChildrenPerParent;
+ // epoch this context is for
+ unsigned short epoch;
+ unsigned short padding;
+
+ static constexpr unsigned char type()
+ {
+ return RESPOND_ANT_EPOCH_CONTEXT;
+ }
+};
+#pragma pack(pop)
+static_assert(sizeof(RespondAntEpochContext) == 120, "RespondAntEpochContext unexpected size");
diff --git a/src/network_messages/broadcast_message.h b/src/network_messages/broadcast_message.h
index fe4c6c5d2..a06a52e94 100644
--- a/src/network_messages/broadcast_message.h
+++ b/src/network_messages/broadcast_message.h
@@ -5,6 +5,7 @@
#define MESSAGE_TYPE_SOLUTION 0
#define MESSAGE_TYPE_CUSTOM_MINING_TASK 1
#define MESSAGE_TYPE_CUSTOM_MINING_SOLUTION 2
+#define MESSAGE_TYPE_ANT_SOLUTION 3
// TODO: documentation needed:
// "A General Message type used to send/receive messages from/to peers." -> right?
diff --git a/src/network_messages/network_message_type.h b/src/network_messages/network_message_type.h
index a9b3993bd..e297d37b8 100644
--- a/src/network_messages/network_message_type.h
+++ b/src/network_messages/network_message_type.h
@@ -51,6 +51,12 @@ enum NetworkMessageType : unsigned char
BROADCAST_CUSTOM_MINING_SOLUTION = 69,
REQUEST_REVENUE_DATA = 70,
RESPOND_REVENUE_DATA = 71,
+ REQUEST_ANT_IDENTITY_TREE = 72,
+ RESPOND_ANT_IDENTITY_TREE = 73,
+ REQUEST_ANT_PARENT_ANN = 74,
+ RESPOND_ANT_PARENT_ANN = 75,
+ REQUEST_ANT_EPOCH_CONTEXT = 76,
+ RESPOND_ANT_EPOCH_CONTEXT = 77,
ORACLE_MACHINE_QUERY = 190, // only on communication channel Core node <-> OM node
ORACLE_MACHINE_REPLY = 191, // only on communication channel Core node <-> OM node
OC_MACHINE_INVOCATION = 192, // only on communication channel Core node <-> OC machine
diff --git a/src/platform/concurrency.h b/src/platform/concurrency.h
index 9bdeb8ff7..072213dd2 100644
--- a/src/platform/concurrency.h
+++ b/src/platform/concurrency.h
@@ -93,6 +93,7 @@ struct LockGuard
// long in windows is 32bits
static_assert(sizeof(long) == 4, "Size of long for _InterlockedExchange is 4 bytes");
#define ATOMIC_STORE32(target, val) _InterlockedExchange((volatile long*)&target, val)
+#define ATOMIC_LOAD32(target) _InterlockedCompareExchange((volatile long*)&target, 0, 0)
#define ATOMIC_INC64(target) _InterlockedIncrement64(&target)
#define ATOMIC_AND64(target, val) _InterlockedAnd64(&target, val)
#define ATOMIC_STORE64(target, val) _InterlockedExchange64(&target, val)
diff --git a/src/public_settings.h b/src/public_settings.h
index b5696078d..fe55126a2 100644
--- a/src/public_settings.h
+++ b/src/public_settings.h
@@ -93,6 +93,13 @@ static unsigned short REVENUE_DATA_END_OF_EPOCH_FILE_NAME[] = L"revenue_data.eoe
static unsigned short REVENUE_DATA_SNAPSHOT_FILE_NAME[] = L"revenue_data.???";
static unsigned short MULTIDIM_REVENUE_SNAPSHOT_FILE_NAME[] = L"revenue_data_multi.???";
static unsigned short MULTIDIM_REVENUE_END_OF_EPOCH_FILE_NAME[] = L"revenue_data_multi.eoe";
+// Ant colony files. The header file carries the meta, the anchor ring and the export set together
+static unsigned short ANT_SNAPSHOT_HEADER_FILENAME[] = L"snapshotAntColonyHeader.???";
+static unsigned short ANT_SNAPSHOT_RECORDS_FILENAME[] = L"snapshotAntColonyRecords.???";
+static unsigned short ANT_SNAPSHOT_POOL_FILENAME[] = L"snapshotAntColonyPool.???";
+static unsigned short ANT_COLONY_REPLAY_CACHE_FILENAME[] = L"antColonyReplayCache.???";
+static unsigned short ANT_COLONY_SOLUTIONS_EOE_FILENAME[] = L"antColonySolutions.eoe";
+static unsigned short ANT_SOL_FLAG_FILE_NAME[] = L"snapshotAntSolutionFlag";
// Neuraxon (even-nonce slot) - reserved for a future algorithm, not yet implemented.
static constexpr unsigned long long NEURAXON_NUMBER_OF_INPUT_NEURONS = 1;
@@ -126,6 +133,23 @@ static constexpr unsigned long long BPP9000_NUMBER_OF_MUTATIONS = 100;
static constexpr unsigned long long BPP9000_NUMBER_OF_WINDOWS = BPP9000_SEQUENCE_LENGTH - BPP9000_WINDOW_WIDTH;
static constexpr unsigned int BPP9000_SOLUTION_THRESHOLD_DEFAULT = 3838;
+// Ant colony: a solution must be published within this many ticks of the anchor its walk seeded from.
+static constexpr unsigned int ANT_PUBLISH_WINDOW_TICKS = 16000;
+
+// Per-parent child cap: a parent accepts at most this many children - a miner's parallel branches
+// off one node. 0 means unbound (no cap). A child's score must still strictly beat its parent's.
+// A child over the cap is rejected without a refund, so miners should stop submitting to a full parent.
+static constexpr unsigned int ANT_MAX_CHILDREN_PER_PARENT = 0;
+
+// Ant colony: tree nodes recorded per epoch; one per accepted solution.
+static constexpr unsigned int ANT_MAX_NODES_PER_EPOCH = 1u << 23;
+
+// Ant colony: replay-cache entries, scores this node already computed so a restart does not
+// recompute them. Node-local, not consensus; a miss only costs time.
+static constexpr unsigned int ANT_REPLAY_CACHE_SIZE = 1u << 20;
+static_assert((ANT_REPLAY_CACHE_SIZE & (ANT_REPLAY_CACHE_SIZE - 1)) == 0,
+ "ANT_REPLAY_CACHE_SIZE must be a power of two, the slot index masks with it");
+
// Multipler of score
static constexpr unsigned int NEURAXON_SOLUTION_MULTIPLER = 1;
static constexpr unsigned int BPP9000_SOLUTION_MULTIPLER = 1;
diff --git a/src/qubic.cpp b/src/qubic.cpp
index 7b5368baa..6dbc35fac 100644
--- a/src/qubic.cpp
+++ b/src/qubic.cpp
@@ -75,10 +75,12 @@
#include "files/files.h"
#include "mining/mining.h"
#include "mining/custom_qubic_mining_storage.h"
+#include "mining/ant_colony/ant_colony_bpp9000.h"
#include "oracle_core/oracle_engine.h"
#include "oracle_core/net_msg_impl.h"
#include "oracle_core/snapshot_files.h"
+#include "mining/ant_colony/ant_pending_solutions.h"
#include "oracle_core/oracle_interfaces_def.h"
#include "qpi/impl/qpi_oracle_impl.h"
@@ -95,8 +97,8 @@
#define CONTRACT_STATES_DEPTH 10 // Is derived from MAX_NUMBER_OF_CONTRACTS (=N)
#define TICK_REQUESTING_PERIOD 500ULL
#define MAX_NUMBER_EPOCH 1000ULL
-#define MAX_NUMBER_OF_MINERS 8192
#define NUMBER_OF_MINER_SOLUTION_FLAGS 0x100000000
+#define NUMBER_OF_ANT_SOLUTION_FLAGS 0x100000000
#define MAX_MESSAGE_PAYLOAD_SIZE MAX_TRANSACTION_SIZE
#define MAX_UNIVERSE_SIZE 1073741824
#define MESSAGE_DISSEMINATION_THRESHOLD 1000000000
@@ -214,8 +216,20 @@ static ScoreFunction<
NUMBER_OF_SOLUTION_PROCESSORS
> * score = nullptr;
static unsigned char* gBpp9000TaskBuffer = nullptr;
+
+// The payload is the { publicKey, miningSeed, nonce }
+static_assert(3 * sizeof(m256i) <= ScoreFunction::TASK_PAYLOAD_MAX,
+ "A legacy solution must fit the task queue payload");
+
+static void scoreLegacySolutionTask(unsigned long long processorNumber, void* payload)
+{
+ const m256i* data = (const m256i*)payload;
+ (*score)(processorNumber, data[0], data[1], data[2]);
+}
+
static volatile char solutionsLock = 0;
static unsigned long long* minerSolutionFlags = NULL;
+static unsigned long long* gAntSolutionFlags = NULL;
static volatile m256i minerPublicKeys[MAX_NUMBER_OF_MINERS + 1];
static volatile unsigned int minerScores[MAX_NUMBER_OF_MINERS + 1];
// Tick in which each miner reached its currently recorded best score, used as ranking tie-breaker
@@ -239,6 +253,9 @@ static constexpr unsigned int gScoreMultiplier[score_engine::AlgoType::MaxAlgoCo
NEURAXON_SOLUTION_MULTIPLER, // Neuraxon (reserved)
BPP9000_SOLUTION_MULTIPLER // Bpp9000
};
+// Bpp9000's score is a raw error count consumed directly by the minimum-is-best ranking; scaling it
+// serves no purpose and a large multiplier would overflow the ranking score. Pin it to 1.
+static_assert(BPP9000_SOLUTION_MULTIPLER == 1, "Bpp9000 error score is ranked by minimum; its multiplier must be 1");
// Active solution threshold for an algorithm
static int getSolutionThreshold(score_engine::AlgoType selectedAlgo)
@@ -253,6 +270,352 @@ static int getSolutionThreshold(score_engine::AlgoType selectedAlgo)
}
static bool applyBpp9000Task();
+static AntColonyBpp9000T gAntColony;
+static AntPendingSolutions gAntPendingSolutions;
+static AntColonyBpp9000T::Ann gAntParentAnnScratch[MAX_NUMBER_OF_PROCESSORS];
+static AntColonyBpp9000T::Ann gAntChildAnnScratch[MAX_NUMBER_OF_PROCESSORS];
+
+#ifndef NDEBUG
+static constexpr unsigned int ANT_DEBUG_PRINTS_PER_EPOCH = 512;
+static unsigned int gAntDebugPrintBudget = ANT_DEBUG_PRINTS_PER_EPOCH;
+static bool antDebugCanPrint()
+{
+ if (gAntDebugPrintBudget == 0)
+ {
+ return false;
+ }
+ gAntDebugPrintBudget--;
+ if (gAntDebugPrintBudget == 0)
+ {
+ logToConsole(L"[ant-colony] debug print budget exhausted, silent until next epoch");
+ }
+ return true;
+}
+#endif
+
+static void antDebugLine(const CHAR16* text)
+{
+#ifndef NDEBUG
+ if (antDebugCanPrint())
+ {
+ logToConsole(text);
+ }
+#endif
+}
+
+static void antDebugPoolDrop(const CHAR16* reason, const AntSolutionBroadcastPayload& payload)
+{
+#ifndef NDEBUG
+ if (!antDebugCanPrint())
+ {
+ return;
+ }
+ CHAR16 msg[256];
+ setText(msg, L"[ant-colony] pool drop ");
+ appendText(msg, reason);
+ appendText(msg, L": parent=");
+ appendNumber(msg, payload.parentTick, FALSE);
+ appendText(msg, L"/");
+ appendNumber(msg, payload.parentSolutionIndexInTick, FALSE);
+ appendText(msg, L" anchor=");
+ appendNumber(msg, payload.anchorTick, FALSE);
+ appendText(msg, L" nonce0=");
+ appendNumber(msg, payload.nonce.m256i_u64[0], FALSE);
+ logToConsole(msg);
+#endif
+}
+
+static void antDebugPending(const CHAR16* outcome, const AntPendingSolution& entry, unsigned int targetTick)
+{
+#ifndef NDEBUG
+ if (!antDebugCanPrint())
+ {
+ return;
+ }
+ CHAR16 msg[256];
+ setText(msg, L"[ant-colony] ");
+ appendText(msg, outcome);
+ appendText(msg, L": parent=");
+ appendNumber(msg, entry.parentRef.tick, FALSE);
+ appendText(msg, L"/");
+ appendNumber(msg, entry.parentRef.solutionIndexInTick, FALSE);
+ appendText(msg, L" anchor=");
+ appendNumber(msg, entry.anchorTick, FALSE);
+ appendText(msg, L" score=");
+ appendNumber(msg, entry.score, FALSE);
+ appendText(msg, L" target=");
+ appendNumber(msg, targetTick, FALSE);
+ logToConsole(msg);
+#endif
+}
+
+static void antDebugAccepted(const AntColonyMiningSolutionTransaction* transaction, unsigned int score,
+ unsigned int depth, unsigned int transactionIndex, ValidityResult result)
+{
+#ifndef NDEBUG
+ if (result != ValidityResult::Valid && result != ValidityResult::ValidNotStored)
+ {
+ return;
+ }
+ if (!antDebugCanPrint())
+ {
+ return;
+ }
+ CHAR16 msg[256];
+ setText(msg, L"[ant-colony] accepted: tick=");
+ appendNumber(msg, system.tick, FALSE);
+ appendText(msg, L" idx=");
+ appendNumber(msg, transactionIndex, FALSE);
+ appendText(msg, L" score=");
+ appendNumber(msg, score, FALSE);
+ appendText(msg, L" depth=");
+ appendNumber(msg, depth, FALSE);
+ appendText(msg, L" stored=");
+ appendNumber(msg, (result == ValidityResult::Valid) ? 1 : 0, FALSE);
+ logToConsole(msg);
+#endif
+}
+
+// Pre-scored ant solutions for the current tick, indexed by TRANSACTION index. Each transaction is
+// enqueued at most once, so no two workers ever write the same slot and no lock is needed
+static bool gAntScoredReady[NUMBER_OF_TRANSACTIONS_PER_TICK];
+static unsigned int gAntScoredValue[NUMBER_OF_TRANSACTIONS_PER_TICK];
+static AntColonyBpp9000T::Ann gAntScoredAnn[NUMBER_OF_TRANSACTIONS_PER_TICK];
+
+// Enqueued per ant solution transaction. 80 bytes, inside ScoreFunction::TASK_PAYLOAD_MAX.
+struct AntScoreTaskPayload
+{
+ m256i pubkey;
+ m256i nonce;
+ SolutionRef parentRef;
+ unsigned int anchorTick;
+ unsigned int txIdx;
+};
+static_assert(sizeof(AntScoreTaskPayload) <= 128, "ant score payload must fit TASK_PAYLOAD_MAX");
+
+// The cache is keyed by what the score is a function of, not by where the inputs were found. A
+// parent named by position and an anchor named by tick number both depend on this node already
+// holding the same tree, which is exactly what a node replaying after a restart is still building.
+static AntColonyBpp9000T::ReplayKey makeAntReplayKey(const m256i& pubkey, const m256i& nonce,
+ const AntColonyBpp9000T::Ann* parentAnn, const m256i& anchorDigest)
+{
+ AntColonyBpp9000T::ReplayKey key;
+ key.pubkey = pubkey;
+ key.nonce = nonce;
+ key.parentAnnHash = m256i::zero(); // a child of the root has no parent network to name
+ if (parentAnn != nullptr)
+ {
+ KangarooTwelve(parentAnn, sizeof(*parentAnn), &key.parentAnnHash, sizeof(key.parentAnnHash));
+ }
+ key.anchorDigest = anchorDigest;
+ return key;
+}
+
+// Score one ant solution ahead of the transaction loop, on whichever processor drains the queue.
+static void scoreAntSolutionTask(unsigned long long processorNumber, void* payload)
+{
+ const AntScoreTaskPayload* task = (const AntScoreTaskPayload*)payload;
+
+ const AntSolutionRecord* parentRec = nullptr;
+ if (gAntColony.tryGetParent(task->parentRef, &parentRec) != ValidityResult::Valid)
+ {
+ return;
+ }
+
+ m256i anchorDigest;
+ if (!gAntColony.getAnchorDigest(task->anchorTick, anchorDigest))
+ {
+ return;
+ }
+
+ const AntColonyBpp9000T::Ann* parentAnn = nullptr;
+ if (parentRec != nullptr)
+ {
+ if (!gAntColony.annOfNonRoot(*parentRec, gAntParentAnnScratch[processorNumber]))
+ {
+ return;
+ }
+ parentAnn = &gAntParentAnnScratch[processorNumber];
+ }
+
+ const AntColonyBpp9000T::ReplayKey replayKey =
+ makeAntReplayKey(task->pubkey, task->nonce, parentAnn, anchorDigest);
+
+ // Check in the cache first if this sol was computed
+ if (!gAntColony.tryGetReplayScore(replayKey, gAntScoredValue[task->txIdx], gAntScoredAnn[task->txIdx]))
+ {
+ // Straight into this transaction's own result slot, so nothing is copied afterwards.
+ gAntScoredValue[task->txIdx] = score->computeAntChildScore(
+ processorNumber, parentAnn, task->pubkey, task->nonce,
+ anchorDigest, gAntScoredAnn[task->txIdx]);
+ gAntColony.putReplayScore(replayKey, gAntScoredValue[task->txIdx], gAntScoredAnn[task->txIdx]);
+ }
+
+ // Last, so the transaction loop never sees a slot whose score or network is half written.
+ gAntScoredReady[task->txIdx] = true;
+}
+
+// The two independent bits an ant solution occupies in gAntSolutionFlags. Hashed over the same
+// triple AntDedupKey uses, so "seen" and "already in the tree" agree on what counts as one solution.
+// Bytes are laid out explicitly rather than hashing a struct, so no padding can make the digest
+// differ between compilers.
+static void computeAntSolutionFlagIndices(const m256i& pubkey, const m256i& nonce,
+ const SolutionRef& parentRef, unsigned int* outFlagIndices)
+{
+ unsigned char preimage[sizeof(m256i) + sizeof(m256i) + sizeof(SolutionRef)];
+ copyMem(preimage, &pubkey, sizeof(pubkey));
+ copyMem(preimage + sizeof(pubkey), &nonce, sizeof(nonce));
+ copyMem(preimage + sizeof(pubkey) + sizeof(nonce), &parentRef, sizeof(parentRef));
+ KangarooTwelve(preimage, sizeof(preimage), outFlagIndices, 2 * sizeof(unsigned int));
+}
+
+// Seen means BOTH bits are set, matching the legacy filter: one bit alone is a collision with some
+// other solution, and requiring two drops the false-positive rate from ~N/2^32 to ~N^2/2^64.
+static bool isAntSolutionSeen(const unsigned int* flagIndices)
+{
+ return (gAntSolutionFlags[flagIndices[0] >> 6] & (1ULL << (flagIndices[0] & 63)))
+ && (gAntSolutionFlags[flagIndices[1] >> 6] & (1ULL << (flagIndices[1] & 63)));
+}
+
+static void markAntSolutionSeen(const unsigned int* flagIndices)
+{
+ gAntSolutionFlags[flagIndices[0] >> 6] |= (1ULL << (flagIndices[0] & 63));
+ gAntSolutionFlags[flagIndices[1] >> 6] |= (1ULL << (flagIndices[1] & 63));
+}
+
+// The anchor digest a solution's mutation walk seeds from, K12(tick || transactionDigest)
+static void computeAntAnchorDigest(unsigned int tick, const m256i& transactionDigest, m256i& out)
+{
+ unsigned char preimage[sizeof(unsigned int) + sizeof(m256i)];
+ copyMem(preimage, &tick, sizeof(tick));
+ copyMem(preimage + sizeof(tick), &transactionDigest, sizeof(transactionDigest));
+ KangarooTwelve(preimage, sizeof(preimage), &out, sizeof(out));
+}
+
+// A pool miner's solution, arriving over BroadcastMessage
+static void queueAntSolution(unsigned long long processorNumber, const m256i& computorPublicKey,
+ const AntSolutionBroadcastPayload& payload)
+{
+ gAntPendingSolutions.noteReceived();
+
+ // A non-canonical nonce is rejected by the scorer without producing a score, so the transaction
+ // would forfeit the deposit with nothing to show. The computor pays that, not the miner.
+ if (!score_engine::ScoreEngineT::isCanonicalAntNonce(payload.nonce.m256i_u8))
+ {
+ antDebugPoolDrop(L"nonCanonical", payload);
+ gAntPendingSolutions.noteDroppedNonCanonical();
+ return;
+ }
+
+ // The same bits the commit path checks before RejectReplay, so a hit here is a solution this
+ // node has already processed on-chain - publishing it again would forfeit the deposit. The
+ // filter is restored from the snapshot, so unlike this buffer the check survives a restart.
+ const SolutionRef parentRef = { payload.parentTick, payload.parentSolutionIndexInTick };
+ unsigned int seenFlagIndices[2];
+ computeAntSolutionFlagIndices(computorPublicKey, payload.nonce, parentRef, seenFlagIndices);
+ if (isAntSolutionSeen(seenFlagIndices))
+ {
+ gAntPendingSolutions.noteDroppedDuplicate();
+ return;
+ }
+
+ // An anchor in the future is malformed, and one the ring no longer holds cannot be scored.
+ m256i anchorDigest;
+ if (payload.anchorTick > system.tick
+ || system.tick - payload.anchorTick > ANT_PUBLISH_WINDOW_TICKS
+ || !gAntColony.getAnchorDigest(payload.anchorTick, anchorDigest))
+ {
+ antDebugPoolDrop(L"badAnchor", payload);
+ gAntPendingSolutions.noteDroppedBadAnchor();
+ return;
+ }
+
+ const AntSolutionRecord* parentRec = nullptr;
+ if (gAntColony.tryGetParent(parentRef, &parentRec) != ValidityResult::Valid)
+ {
+ antDebugPoolDrop(L"parentUnknown", payload);
+ gAntPendingSolutions.noteDroppedParentUnknown();
+ return;
+ }
+
+ const score_engine::ScoreBpp9000T::ANN* parentAnn = nullptr;
+ if (parentRec != nullptr)
+ {
+ if (!gAntColony.annOfNonRoot(*parentRec, gAntParentAnnScratch[processorNumber]))
+ {
+ antDebugPoolDrop(L"parentUnknown", payload);
+ gAntPendingSolutions.noteDroppedParentUnknown();
+ return;
+ }
+ parentAnn = &gAntParentAnnScratch[processorNumber];
+ }
+
+ // Building the key is far cheaper than a miss, so the cache is consulted first.
+ const AntColonyBpp9000T::ReplayKey replayKey =
+ makeAntReplayKey(computorPublicKey, payload.nonce, parentAnn, anchorDigest);
+ unsigned int childScore = 0;
+ if (!gAntColony.tryGetReplayScore(replayKey, childScore, gAntChildAnnScratch[processorNumber]))
+ {
+ childScore = score->computeAntChildScore(processorNumber, parentAnn, computorPublicKey,
+ payload.nonce, anchorDigest, gAntChildAnnScratch[processorNumber]);
+ // Cached whatever the outcome: a timed-out network scores invalid, and the walk is
+ // deterministic, so the cached rejection stays right - without the entry the same doomed
+ // solution costs a full walk again on every path that sees it.
+ gAntColony.putReplayScore(replayKey, childScore, gAntChildAnnScratch[processorNumber]);
+ }
+ if (!score->isValidScore(childScore, score_engine::AlgoType::Bpp9000))
+ {
+ antDebugPoolDrop(L"unscorable", payload);
+ gAntPendingSolutions.noteDroppedUnscorable();
+ return;
+ }
+
+ // The sender's own number, checked where it can still prevent work rather than merely be counted.
+ if (payload.claimedScore != childScore)
+ {
+ antDebugPoolDrop(L"claimMismatch", payload);
+ gAntPendingSolutions.noteClaimMismatch();
+ return;
+ }
+
+ // The child count only grows, so passing now is not a promise it will pass at publication - the
+ // publisher re-checks. Failing now is final enough to refuse the slot.
+ const unsigned int childCount = gAntColony.childCountForQuery(parentRef, computorPublicKey);
+ const ChildCandidate candidate{ computorPublicKey, childScore, payload.anchorTick, system.tick };
+ if (AntColonyBpp9000T::validateChild(candidate, parentRec, childCount,
+ gAntColony.errorThreshold()) != ValidityResult::Valid)
+ {
+ antDebugPoolDrop(L"unacceptable", payload);
+ gAntPendingSolutions.noteDroppedUnacceptable();
+ return;
+ }
+
+ gAntPendingSolutions.add(computorPublicKey, parentRef, payload.anchorTick, childScore,
+ payload.nonce);
+}
+
+// Reseed the colony for a new epoch. The root seed is score->currentRandomSeed, the epoch-start
+// spectrum digest
+static void antColonyBeginEpoch()
+{
+#ifndef NDEBUG
+ gAntDebugPrintBudget = ANT_DEBUG_PRINTS_PER_EPOCH;
+#endif
+ gAntPendingSolutions.reset();
+ gAntColony.beginEpoch(score->currentRandomSeed, system.initialTick);
+ gAntColony.setErrorThreshold((unsigned int)getSolutionThreshold(score_engine::AlgoType::Bpp9000));
+
+ // Every identity's root derives from this one value, so a node that seeded differently builds a
+ // different forest and diverges. Logged as an identity so operators can compare it across nodes
+ // by eye at epoch start, which is cheaper than finding out from a digest split later.
+ CHAR16 digestChars[60 + 1];
+ getIdentity(gAntColony.rootSeed().m256i_u8, digestChars, true);
+ CHAR16 msg[128];
+ setText(msg, L"[ant-colony] Root seed = ");
+ appendText(msg, digestChars);
+ logToConsole(msg);
+}
+
// DOGE merged-mining shares
static volatile char gDogeMiningSharesCountLock = 0;
static unsigned int gDogeMiningSharesCount[NUMBER_OF_COMPUTORS] = { 0 };
@@ -623,6 +986,12 @@ static void processBroadcastMessage(const unsigned long long processorNumber, Re
const m256i& solution_miningSeed = *(m256i*)((unsigned char*)request + sizeof(BroadcastMessage));
const m256i& solution_nonce = *(m256i*)((unsigned char*)request + sizeof(BroadcastMessage) + 32);
+ // standalone mining disabled for bpp9000
+ if (score_engine::getAlgoType(solution_nonce.m256i_u8) == score_engine::AlgoType::Bpp9000)
+ {
+ break;
+ }
+
const unsigned int solution_claimedScore = *(unsigned int*)((unsigned char*)request + sizeof(BroadcastMessage) + 64);
unsigned int k;
for (k = 0; k < system.numberOfSolutions; k++)
@@ -669,6 +1038,19 @@ static void processBroadcastMessage(const unsigned long long processorNumber, Re
}
}
break;
+
+ case MESSAGE_TYPE_ANT_SOLUTION:
+ {
+ // Exact size, not a minimum: the payload is fixed, so a longer
+ // one is a different message rather than a forward-compatible
+ // variant of this one.
+ if (messagePayloadSize == sizeof(AntSolutionBroadcastPayload))
+ {
+ queueAntSolution(processorNumber, request->destinationPublicKey,
+ *(AntSolutionBroadcastPayload*)((unsigned char*)request + sizeof(BroadcastMessage)));
+ }
+ }
+ break;
}
}
}
@@ -1019,6 +1401,52 @@ static void processBroadcastTransaction(Peer* peer, RequestResponseHeader* heade
}
}
+ // Same latency hiding for ant solution transactions, we do simple check first then the last
+ // is the score engine that where the heavy load stay
+ if (preprocessSolutionFlags[processorNumber]
+ && AntColonyMiningSolutionTransaction::isSolutionTransaction(request))
+ {
+ const AntColonyMiningSolutionTransaction* antTx = (const AntColonyMiningSolutionTransaction*)request;
+ const SolutionRef preParentRef = { antTx->parentTick, antTx->parentSolutionIndexInTick };
+ unsigned int preFlagIndices[2];
+ computeAntSolutionFlagIndices(antTx->sourcePublicKey, antTx->nonce, preParentRef, preFlagIndices);
+ const int spectrumIdx = spectrumIndex(antTx->sourcePublicKey);
+ if (spectrumIdx >= 0
+ && energy(spectrumIdx) >= AntColonyMiningSolutionTransaction::minAmount()
+ && !isAntSolutionSeen(preFlagIndices)
+ && score_engine::ScoreEngineT::isCanonicalAntNonce(antTx->nonce.m256i_u8))
+ {
+ const AntSolutionRecord* preParentRec = nullptr;
+ m256i preAnchorDigest;
+ if (gAntColony.tryGetParent(preParentRef, &preParentRec) == ValidityResult::Valid
+ && gAntColony.getAnchorDigest(antTx->anchorTick, preAnchorDigest))
+ {
+ const AntColonyBpp9000T::Ann* preParentAnn = nullptr;
+ bool preParentOk = true;
+ if (preParentRec != nullptr)
+ {
+ preParentOk = gAntColony.annOfNonRoot(*preParentRec, gAntParentAnnScratch[processorNumber]);
+ preParentAnn = &gAntParentAnnScratch[processorNumber];
+ }
+ if (preParentOk)
+ {
+ const AntColonyBpp9000T::ReplayKey preKey = makeAntReplayKey(
+ antTx->sourcePublicKey, antTx->nonce, preParentAnn, preAnchorDigest);
+ unsigned int preScore = 0;
+ if (!gAntColony.tryGetReplayScore(preKey, preScore, gAntChildAnnScratch[processorNumber]))
+ {
+ preScore = score->computeAntChildScore(processorNumber, preParentAnn,
+ antTx->sourcePublicKey, antTx->nonce, preAnchorDigest,
+ gAntChildAnnScratch[processorNumber]);
+ // cache this score so later can skip the heavy score computation,
+ // invalid ones included
+ gAntColony.putReplayScore(preKey, preScore, gAntChildAnnScratch[processorNumber]);
+ }
+ }
+ }
+ }
+ }
+
// shortcut: oracle reply reveal transactions are analyzed immediately after receiving them (before execution of the tx),
// in order to minimize the number of reveal transaction (one per oracle query is enough, so no reveal tx is generated
// after one has been seen)
@@ -1359,6 +1787,171 @@ static void processRequestContractFunction(Peer* peer, const unsigned long long
}
}
+// One response buffer per processor
+static AntIdentityTreeResponse gAntIdentityTreeResponseBuffer[MAX_NUMBER_OF_PROCESSORS];
+
+struct AntParentAnnResponse
+{
+ RespondAntParentAnnHeader header;
+ AntColonyBpp9000T::Ann ann;
+};
+static_assert(sizeof(AntParentAnnResponse)
+ == sizeof(RespondAntParentAnnHeader) + sizeof(AntColonyBpp9000T::Ann),
+ "AntParentAnnResponse must have no padding between the header and the network");
+static AntParentAnnResponse gAntParentAnnResponseBuffer[MAX_NUMBER_OF_PROCESSORS];
+
+// Request ant colony in epoch contex
+static void processRequestAntEpochContext(Peer* peer, RequestResponseHeader* header)
+{
+ RespondAntEpochContext respond;
+ setMem(&respond, sizeof(respond), 0);
+
+ respond.spectrumDigest = gAntColony.rootSeed();
+ respond.threshold = gAntColony.errorThreshold();
+ respond.freshnessWindow = ANT_PUBLISH_WINDOW_TICKS;
+ respond.solutionCount = gAntColony.solutionCount();
+ respond.freeAnnSlotsCount = gAntColony.freeAnnSlotsCount();
+ respond.maxChildrenPerParent = ANT_MAX_CHILDREN_PER_PARENT;
+ respond.epoch = system.epoch;
+ respond.topologyHash = *(const m256i*)BPP9000_TOPOLOGY_HASH;
+ respond.dataHash = *(const m256i*)BPP9000_DATA_HASH;
+
+ enqueueResponse(peer, sizeof(respond), RespondAntEpochContext::type(), header->dejavu(), &respond);
+}
+
+// The parents ONE identity can branch from, each with the bar a child of it must beat. Scoped by
+// pubkey: a child must name a parent in its own tree, so an unscoped answer would be mostly nodes the
+// caller can never use.
+//
+// Paged, because the store holds millions and a response is one datagram - the cursor is a record
+// index, and ANT_IDENTITY_TREE_SCAN_BUDGET caps how far one request may scan, so a caller cannot
+// walk the whole store in a single call. Sweeping a full store therefore costs many requests; the
+// alternative, walking the identity's tree through the head maps, needs a resumable cursor that does
+// not fit a record index, and the scan is cache-friendly where a chain walk is not.
+//
+// Operator-signed
+static void processRequestAntIdentityTree(unsigned long long processorNumber, Peer* peer, RequestResponseHeader* header)
+{
+ if (processorNumber >= MAX_NUMBER_OF_PROCESSORS)
+ {
+ return;
+ }
+ if (header->size() != sizeof(RequestResponseHeader) + sizeof(RequestAntIdentityTree) + SIGNATURE_SIZE)
+ {
+ return;
+ }
+ const RequestAntIdentityTree* request = header->getPayload();
+
+ // Signature check
+ unsigned char digest[32];
+ KangarooTwelve(request, header->size() - sizeof(RequestResponseHeader) - SIGNATURE_SIZE, digest, sizeof(digest));
+ if (!verify(operatorPublicKey.m256i_u8, digest, ((const unsigned char*)header + (header->size() - SIGNATURE_SIZE))))
+ {
+ antDebugLine(L"[ant-colony] query signature rejected");
+ return;
+ }
+
+ AntIdentityTreeResponse& response = gAntIdentityTreeResponseBuffer[processorNumber];
+ setMem(&response, sizeof(response), 0);
+ response.header.itemSize = (unsigned int)sizeof(AntIdentityTreeNode);
+
+ const unsigned int total = gAntColony.solutionCount();
+
+ unsigned int idx = request->fromIndex;
+ unsigned int scanned = 0;
+ while (idx < total
+ && response.header.count < ANT_IDENTITY_TREE_NODES_PER_RESPONSE
+ && scanned < ANT_IDENTITY_TREE_SCAN_BUDGET)
+ {
+ const AntSolutionRecord* rec = gAntColony.recordAt(idx);
+ if (rec == nullptr)
+ {
+ break;
+ }
+ scanned++;
+ if (!(rec->pubkey == request->pubkey))
+ {
+ idx++;
+ continue;
+ }
+
+ AntIdentityTreeNode& item = response.items[response.header.count];
+ item.selfTick = rec->selfRef.tick;
+ item.selfSolutionIndexInTick = rec->selfRef.solutionIndexInTick;
+ item.parentTick = rec->parentRef.tick;
+ item.parentSolutionIndexInTick = rec->parentRef.solutionIndexInTick;
+ item.score = rec->score;
+ item.childCount = gAntColony.childCountForQuery(rec->selfRef, rec->pubkey);
+ item.anchorTick = rec->anchorTick;
+ item.depth = rec->depth;
+ response.header.count++;
+ idx++;
+ }
+
+ // Zero means the caller reached the end of what this node holds, per the protocol.
+ response.header.nextIndex = (idx < total) ? idx : 0;
+
+ enqueueResponse(peer,
+ (unsigned int)sizeof(response.header) + response.header.count * (unsigned int)sizeof(AntIdentityTreeNode),
+ RespondAntIdentityTreeHeader::type(), header->dejavu(), &response);
+}
+
+// One stored node's network, for the pool that is about to mine a child of it
+static void processRequestAntParentAnn(unsigned long long processorNumber, Peer* peer, RequestResponseHeader* header)
+{
+ if (processorNumber >= MAX_NUMBER_OF_PROCESSORS)
+ {
+ return;
+ }
+ if (header->size() != sizeof(RequestResponseHeader) + sizeof(RequestAntParentAnn) + SIGNATURE_SIZE)
+ {
+ return;
+ }
+ const RequestAntParentAnn* request = header->getPayload();
+
+ // Signature check
+ unsigned char digest[32];
+ KangarooTwelve(request, header->size() - sizeof(RequestResponseHeader) - SIGNATURE_SIZE, digest, sizeof(digest));
+ if (!verify(operatorPublicKey.m256i_u8, digest, ((const unsigned char*)header + (header->size() - SIGNATURE_SIZE))))
+ {
+ antDebugLine(L"[ant-colony] query signature rejected");
+ return;
+ }
+
+ AntParentAnnResponse& response = gAntParentAnnResponseBuffer[processorNumber];
+ setMem(&response, sizeof(response), 0);
+ response.header.parentRefTick = request->parentRefTick;
+ response.header.parentRefSolutionIndexInTick = request->parentRefSolutionIndexInTick;
+
+ const SolutionRef ref = { request->parentRefTick, request->parentRefSolutionIndexInTick };
+ if (ref.isRoot())
+ {
+ // Roots are never stored; the miner derives its own from the epoch context's seed.
+ response.header.status = ANT_PARENT_ANN_STATUS_IS_ROOT;
+ enqueueResponse(peer, sizeof(response.header), RespondAntParentAnnHeader::type(), header->dejavu(), &response);
+ return;
+ }
+
+ const AntSolutionRecord* rec = nullptr;
+ const ValidityResult parentResult = gAntColony.tryGetParent(ref, &rec);
+ bool annLoaded = false;
+ if (parentResult == ValidityResult::Valid && rec != nullptr)
+ {
+ annLoaded = gAntColony.annOfNonRoot(*rec, response.ann);
+ }
+ if (!annLoaded)
+ {
+ response.header.status = ANT_PARENT_ANN_STATUS_NOT_FOUND;
+ enqueueResponse(peer, sizeof(response.header), RespondAntParentAnnHeader::type(), header->dejavu(), &response);
+ return;
+ }
+
+ response.header.status = ANT_PARENT_ANN_STATUS_OK;
+ response.header.annSizeBytes = (unsigned int)sizeof(response.ann);
+ enqueueResponse(peer, (unsigned int)(sizeof(response.header) + sizeof(response.ann)),
+ RespondAntParentAnnHeader::type(), header->dejavu(), &response);
+}
+
static void processRequestSystemInfo(Peer* peer, RequestResponseHeader* header)
{
RespondSystemInfo respondedSystemInfo;
@@ -1806,6 +2399,7 @@ static void checkAndSwitchMiningPhase(short tickEpoch, TimeDate tickDate, bool r
if (resetPhase)
{
setNewMiningSeed();
+ antColonyBeginEpoch();
}
// Roll DOGE per-phase stats at broadcast-cycle boundaries (display only).
@@ -1896,7 +2490,7 @@ static void requestProcessor(void* ProcedureArgument)
if (solutionProcessorFlags[processorNumber])
{
PROFILE_NAMED_SCOPE("requestProcessor(): solution processing");
- score->tryProcessSolution(processorNumber);
+ score->tryProcessOneTask(processorNumber);
}
if (requestQueueElementTail == requestQueueElementHead)
@@ -2131,6 +2725,24 @@ static void requestProcessor(void* ProcedureArgument)
}
break;
+ case RequestAntEpochContext::type():
+ {
+ processRequestAntEpochContext(peer, header);
+ }
+ break;
+
+ case RequestAntIdentityTree::type():
+ {
+ processRequestAntIdentityTree(processorNumber, peer, header);
+ }
+ break;
+
+ case RequestAntParentAnn::type():
+ {
+ processRequestAntParentAnn(processorNumber, peer, header);
+ }
+ break;
+
#if ADDON_TX_STATUS_REQUEST
/* qli: process RequestTxStatus message */
case RequestTxStatus::type():
@@ -2476,6 +3088,145 @@ static bool ranksBelow(unsigned int scoreA, unsigned int tickA, unsigned int sco
return tickA > tickB;
}
+// Tick-processor only
+static void updateMinerRankingAndFutureComputors(
+ const m256i& sourcePublicKey,
+ unsigned int newScore,
+ unsigned int newTick)
+{
+ ACQUIRE(minerScoreArrayLock);
+ bool minerEntryChanged = false;
+ unsigned int minerIndex;
+ for (minerIndex = 0; minerIndex < numberOfMiners; minerIndex++)
+ {
+ if (sourcePublicKey == minerPublicKeys[minerIndex])
+ {
+ if (newScore < minerScores[minerIndex])
+ {
+ minerScores[minerIndex] = newScore;
+ minerBestScoreTicks[minerIndex] = newTick;
+ minerEntryChanged = true;
+ }
+
+ break;
+ }
+ }
+ if (minerIndex == numberOfMiners)
+ {
+ if (numberOfMiners < MAX_NUMBER_OF_MINERS)
+ {
+ minerPublicKeys[numberOfMiners] = sourcePublicKey;
+ minerBestScoreTicks[numberOfMiners] = newTick;
+ minerScores[numberOfMiners++] = newScore;
+ minerEntryChanged = true;
+ }
+ else
+ {
+ // The table is full. Entries beyond the computor block are kept sorted, so the
+ // worst-ranked one sits at the end and is replaced only if the newcomer outranks it.
+ const unsigned int worstIndex = numberOfMiners - 1;
+ if (ranksBelow(minerScores[worstIndex], minerBestScoreTicks[worstIndex], newScore, newTick))
+ {
+ minerPublicKeys[worstIndex] = sourcePublicKey;
+ minerScores[worstIndex] = newScore;
+ minerBestScoreTicks[worstIndex] = newTick;
+ minerIndex = worstIndex;
+ minerEntryChanged = true;
+ }
+ }
+ }
+
+ if (minerEntryChanged)
+ {
+ const m256i tmpPublicKey = minerPublicKeys[minerIndex];
+ const unsigned int tmpScore = minerScores[minerIndex];
+ const unsigned int tmpTick = minerBestScoreTicks[minerIndex];
+ while (minerIndex > (unsigned int)(minerIndex < NUMBER_OF_COMPUTORS ? 0 : NUMBER_OF_COMPUTORS)
+ && ranksBelow(minerScores[minerIndex - 1], minerBestScoreTicks[minerIndex - 1], minerScores[minerIndex], minerBestScoreTicks[minerIndex]))
+ {
+ minerPublicKeys[minerIndex] = minerPublicKeys[minerIndex - 1];
+ minerScores[minerIndex] = minerScores[minerIndex - 1];
+ minerBestScoreTicks[minerIndex] = minerBestScoreTicks[minerIndex - 1];
+ minerPublicKeys[--minerIndex] = tmpPublicKey;
+ minerScores[minerIndex] = tmpScore;
+ minerBestScoreTicks[minerIndex] = tmpTick;
+ }
+ }
+
+ // combine 225 worst current computors with 225 best candidates
+ for (unsigned int i = 0; i < NUMBER_OF_COMPUTORS - QUORUM; i++)
+ {
+ competitorPublicKeys[i] = minerPublicKeys[QUORUM + i];
+ competitorScores[i] = minerScores[QUORUM + i];
+ competitorTicks[i] = minerBestScoreTicks[QUORUM + i];
+ competitorComputorStatuses[i] = true;
+
+ if (NUMBER_OF_COMPUTORS + i < numberOfMiners)
+ {
+ competitorPublicKeys[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerPublicKeys[NUMBER_OF_COMPUTORS + i];
+ competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerScores[NUMBER_OF_COMPUTORS + i];
+ competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerBestScoreTicks[NUMBER_OF_COMPUTORS + i];
+ }
+ else
+ {
+ competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = NO_MINER_SCORE;
+ competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = 0;
+ }
+ competitorComputorStatuses[i + (NUMBER_OF_COMPUTORS - QUORUM)] = false;
+ }
+ RELEASE(minerScoreArrayLock);
+
+ // bubble sorting -> top 225 from competitorPublicKeys have computors and candidates which are the best from that subset
+ for (unsigned int i = NUMBER_OF_COMPUTORS - QUORUM; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++)
+ {
+ int j = i;
+ const m256i tmpPublicKey = competitorPublicKeys[j];
+ const unsigned int tmpScore = competitorScores[j];
+ const unsigned int tmpTick = competitorTicks[j];
+ const bool tmpComputorStatus = false;
+ while (j
+ && ranksBelow(competitorScores[j - 1], competitorTicks[j - 1], competitorScores[j], competitorTicks[j]))
+ {
+ competitorPublicKeys[j] = competitorPublicKeys[j - 1];
+ competitorScores[j] = competitorScores[j - 1];
+ competitorTicks[j] = competitorTicks[j - 1];
+ competitorComputorStatuses[j] = competitorComputorStatuses[j - 1];
+ competitorPublicKeys[--j] = tmpPublicKey;
+ competitorScores[j] = tmpScore;
+ competitorTicks[j] = tmpTick;
+ competitorComputorStatuses[j] = tmpComputorStatus;
+ }
+ }
+
+ minimumComputorScore = competitorScores[NUMBER_OF_COMPUTORS - QUORUM - 1];
+
+ unsigned char candidateCounter = 0;
+ for (unsigned int i = 0; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++)
+ {
+ if (!competitorComputorStatuses[i])
+ {
+ minimumCandidateScore = competitorScores[i];
+ candidateCounter++;
+ }
+ }
+ if (candidateCounter < NUMBER_OF_COMPUTORS - QUORUM)
+ {
+ minimumCandidateScore = minimumComputorScore;
+ }
+
+ ACQUIRE(minerScoreArrayLock);
+ for (unsigned int i = 0; i < QUORUM; i++)
+ {
+ system.futureComputors[i] = minerPublicKeys[i];
+ }
+ RELEASE(minerScoreArrayLock);
+
+ for (unsigned int i = QUORUM; i < NUMBER_OF_COMPUTORS; i++)
+ {
+ system.futureComputors[i] = competitorPublicKeys[i - QUORUM];
+ }
+}
+
static void processTickTransactionSolution(const MiningSolutionTransaction* transaction, const unsigned long long processorNumber)
{
PROFILE_SCOPE();
@@ -2488,6 +3239,11 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran
ASSERT(transaction->amount >=MiningSolutionTransaction::minAmount()
&& transaction->inputSize == MiningSolutionTransaction::minInputSize()
&& transaction->inputType == MiningSolutionTransaction::transactionType());
+ // Standalone mining is disabled; bpp9000 is mined through the ant colony only.
+ if (score_engine::getAlgoType(transaction->nonce.m256i_u8) == score_engine::AlgoType::Bpp9000)
+ {
+ return;
+ }
m256i data[3] = { transaction->sourcePublicKey, transaction->miningSeed, transaction->nonce };
static_assert(sizeof(data) == 3 * 32, "Unexpected array size");
@@ -2557,140 +3313,9 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran
// accepted solutions
const unsigned int newScore = solutionScore * gScoreMultiplier[selectedAlgo];
const unsigned int newTick = system.tick;
-
- ACQUIRE(minerScoreArrayLock);
- bool minerEntryChanged = false;
- unsigned int minerIndex;
- for (minerIndex = 0; minerIndex < numberOfMiners; minerIndex++)
- {
- if (transaction->sourcePublicKey == minerPublicKeys[minerIndex])
- {
- if (newScore < minerScores[minerIndex])
- {
- minerScores[minerIndex] = newScore;
- minerBestScoreTicks[minerIndex] = newTick;
- minerEntryChanged = true;
- }
-
- break;
- }
- }
- if (minerIndex == numberOfMiners)
- {
- if (numberOfMiners < MAX_NUMBER_OF_MINERS)
- {
- minerPublicKeys[numberOfMiners] = transaction->sourcePublicKey;
- minerBestScoreTicks[numberOfMiners] = newTick;
- minerScores[numberOfMiners++] = newScore;
- minerEntryChanged = true;
- }
- else
- {
- // The table is full. Entries beyond the computor block are kept sorted, so the
- // worst-ranked one sits at the end and is replaced only if the newcomer outranks it.
- const unsigned int worstIndex = numberOfMiners - 1;
- if (ranksBelow(minerScores[worstIndex], minerBestScoreTicks[worstIndex], newScore, newTick))
- {
- minerPublicKeys[worstIndex] = transaction->sourcePublicKey;
- minerScores[worstIndex] = newScore;
- minerBestScoreTicks[worstIndex] = newTick;
- minerIndex = worstIndex;
- minerEntryChanged = true;
- }
- }
- }
-
- if (minerEntryChanged)
- {
- const m256i tmpPublicKey = minerPublicKeys[minerIndex];
- const unsigned int tmpScore = minerScores[minerIndex];
- const unsigned int tmpTick = minerBestScoreTicks[minerIndex];
- while (minerIndex > (unsigned int)(minerIndex < NUMBER_OF_COMPUTORS ? 0 : NUMBER_OF_COMPUTORS)
- && ranksBelow(minerScores[minerIndex - 1], minerBestScoreTicks[minerIndex - 1], minerScores[minerIndex], minerBestScoreTicks[minerIndex]))
- {
- minerPublicKeys[minerIndex] = minerPublicKeys[minerIndex - 1];
- minerScores[minerIndex] = minerScores[minerIndex - 1];
- minerBestScoreTicks[minerIndex] = minerBestScoreTicks[minerIndex - 1];
- minerPublicKeys[--minerIndex] = tmpPublicKey;
- minerScores[minerIndex] = tmpScore;
- minerBestScoreTicks[minerIndex] = tmpTick;
- }
- }
-
- // combine 225 worst current computors with 225 best candidates
- for (unsigned int i = 0; i < NUMBER_OF_COMPUTORS - QUORUM; i++)
- {
- competitorPublicKeys[i] = minerPublicKeys[QUORUM + i];
- competitorScores[i] = minerScores[QUORUM + i];
- competitorTicks[i] = minerBestScoreTicks[QUORUM + i];
- competitorComputorStatuses[i] = true;
-
- if (NUMBER_OF_COMPUTORS + i < numberOfMiners)
- {
- competitorPublicKeys[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerPublicKeys[NUMBER_OF_COMPUTORS + i];
- competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerScores[NUMBER_OF_COMPUTORS + i];
- competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = minerBestScoreTicks[NUMBER_OF_COMPUTORS + i];
- }
- else
- {
- competitorScores[i + (NUMBER_OF_COMPUTORS - QUORUM)] = NO_MINER_SCORE;
- competitorTicks[i + (NUMBER_OF_COMPUTORS - QUORUM)] = 0;
- }
- competitorComputorStatuses[i + (NUMBER_OF_COMPUTORS - QUORUM)] = false;
- }
- RELEASE(minerScoreArrayLock);
-
- // bubble sorting -> top 225 from competitorPublicKeys have computors and candidates which are the best from that subset
- for (unsigned int i = NUMBER_OF_COMPUTORS - QUORUM; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++)
- {
- int j = i;
- const m256i tmpPublicKey = competitorPublicKeys[j];
- const unsigned int tmpScore = competitorScores[j];
- const unsigned int tmpTick = competitorTicks[j];
- const bool tmpComputorStatus = false;
- while (j
- && ranksBelow(competitorScores[j - 1], competitorTicks[j - 1], competitorScores[j], competitorTicks[j]))
- {
- competitorPublicKeys[j] = competitorPublicKeys[j - 1];
- competitorScores[j] = competitorScores[j - 1];
- competitorTicks[j] = competitorTicks[j - 1];
- competitorComputorStatuses[j] = competitorComputorStatuses[j - 1];
- competitorPublicKeys[--j] = tmpPublicKey;
- competitorScores[j] = tmpScore;
- competitorTicks[j] = tmpTick;
- competitorComputorStatuses[j] = tmpComputorStatus;
- }
- }
-
- minimumComputorScore = competitorScores[NUMBER_OF_COMPUTORS - QUORUM - 1];
-
- unsigned char candidateCounter = 0;
- for (unsigned int i = 0; i < (NUMBER_OF_COMPUTORS - QUORUM) * 2; i++)
- {
- if (!competitorComputorStatuses[i])
- {
- minimumCandidateScore = competitorScores[i];
- candidateCounter++;
- }
- }
- if (candidateCounter < NUMBER_OF_COMPUTORS - QUORUM)
- {
- minimumCandidateScore = minimumComputorScore;
- }
-
- ACQUIRE(minerScoreArrayLock);
- for (unsigned int i = 0; i < QUORUM; i++)
- {
- system.futureComputors[i] = minerPublicKeys[i];
- }
- RELEASE(minerScoreArrayLock);
-
- for (unsigned int i = QUORUM; i < NUMBER_OF_COMPUTORS; i++)
- {
- system.futureComputors[i] = competitorPublicKeys[i - QUORUM];
- }
+ updateMinerRankingAndFutureComputors(transaction->sourcePublicKey, newScore, newTick);
}
- }
+ }
}
else
{
@@ -2730,6 +3355,171 @@ static void processTickTransactionSolution(const MiningSolutionTransaction* tran
}
}
+// One ant solution: resolve its parent, score the child against it, and commit. Every rejection
+// forfeits the deposit by simply not refunding it
+// One line per ant solution transaction, whatever became of it. The body follows the logger's own
+// convention: compiled out with LOG_CUSTOM_MESSAGES, so a build without qlogging pays nothing here.
+static void logAntSolutionOutcome(const AntColonyMiningSolutionTransaction* transaction,
+ unsigned int score, ValidityResult result)
+{
+#if LOG_CUSTOM_MESSAGES
+ AntSolutionLogMessage logMsg;
+ logMsg._type = CUSTOM_MESSAGE_ANT_SOLUTION;
+ logMsg.sourcePublicKey = transaction->sourcePublicKey;
+ logMsg.nonce = transaction->nonce;
+ logMsg.parentTick = transaction->parentTick;
+ logMsg.parentSolutionIndexInTick = transaction->parentSolutionIndexInTick;
+ logMsg.anchorTick = transaction->anchorTick;
+ logMsg.score = score;
+ logMsg.result = (unsigned int)result;
+ logger.logCustomMessage(logMsg);
+#endif
+}
+
+static void processTickTransactionAntColonySolution(
+ const AntColonyMiningSolutionTransaction* transaction,
+ unsigned int transactionIndex,
+ const unsigned long long processorNumber)
+{
+ AntColonyBpp9000T::Ann& parentAnnScratch = gAntParentAnnScratch[processorNumber];
+ AntColonyBpp9000T::Ann& childAnnScratch = gAntChildAnnScratch[processorNumber];
+
+ const SolutionRef parentRef = { transaction->parentTick, transaction->parentSolutionIndexInTick };
+
+ // Already looked at, accepted or not. Marked BEFORE the walk below, so one solution costs at
+ // most one walk for the whole epoch - _dedup alone would only cover the accepted ones.
+ unsigned int antFlagIndices[2];
+ computeAntSolutionFlagIndices(transaction->sourcePublicKey, transaction->nonce, parentRef, antFlagIndices);
+ if (isAntSolutionSeen(antFlagIndices))
+ {
+ gAntColony.recordReject(ValidityResult::RejectReplay);
+ logAntSolutionOutcome(transaction, 0, ValidityResult::RejectReplay);
+ return;
+ }
+ markAntSolutionSeen(antFlagIndices);
+
+ // Reject a parent ref into the current or a later tick: a real parent is always on-chain from an
+ // earlier tick. Root is exempt, it is derived rather than stored.
+ if (!parentRef.isRoot() && parentRef.tick >= system.tick)
+ {
+ gAntColony.recordReject(ValidityResult::RejectParentNotRegistered);
+ logAntSolutionOutcome(transaction, 0, ValidityResult::RejectParentNotRegistered);
+ return;
+ }
+
+ const AntSolutionRecord* parentRec = nullptr;
+ ValidityResult result = gAntColony.tryGetParent(parentRef, &parentRec);
+ if (result != ValidityResult::Valid)
+ {
+ gAntColony.recordReject(result);
+ logAntSolutionOutcome(transaction, 0, result);
+ return;
+ }
+
+ // The anchor digest seeds the child's mutation walk, so an anchor the ring no longer holds cannot
+ // be scored
+ m256i anchorDigest;
+ if (!gAntColony.getAnchorDigest(transaction->anchorTick, anchorDigest))
+ {
+ gAntColony.recordReject(ValidityResult::RejectStale);
+ logAntSolutionOutcome(transaction, 0, ValidityResult::RejectStale);
+ return;
+ }
+
+ unsigned int childScore;
+ const AntColonyBpp9000T::Ann* childAnn;
+ if (gAntScoredReady[transactionIndex])
+ {
+ childScore = gAntScoredValue[transactionIndex];
+ childAnn = &gAntScoredAnn[transactionIndex];
+ }
+ else
+ {
+ // A null parent record means root, the scorer derives the submitter's own root, since roots
+ // are never stored and so cannot be handed in.
+ const AntColonyBpp9000T::Ann* parentAnn = nullptr;
+ if (parentRec != nullptr)
+ {
+ if (!gAntColony.annOfNonRoot(*parentRec, parentAnnScratch))
+ {
+ gAntColony.recordReject(ValidityResult::RejectParentNotRegistered);
+ logAntSolutionOutcome(transaction, 0, ValidityResult::RejectParentNotRegistered);
+ return;
+ }
+ parentAnn = &parentAnnScratch;
+ }
+
+ // Same cache the async path uses. Reached when the pre-scan did not enqueue this one or the
+ // queue did not drain in time, which is exactly the catch-up case the cache exists for.
+ const AntColonyBpp9000T::ReplayKey replayKey =
+ makeAntReplayKey(transaction->sourcePublicKey, transaction->nonce, parentAnn, anchorDigest);
+ if (!gAntColony.tryGetReplayScore(replayKey, childScore, childAnnScratch))
+ {
+ childScore = score->computeAntChildScore(
+ processorNumber, parentAnn, transaction->sourcePublicKey, transaction->nonce,
+ anchorDigest, childAnnScratch);
+ gAntColony.putReplayScore(replayKey, childScore, childAnnScratch);
+ }
+ childAnn = &childAnnScratch;
+ }
+
+ if (!score->isValidScore(childScore, score_engine::AlgoType::Bpp9000))
+ {
+ gAntColony.recordReject(ValidityResult::RejectNonCanonicalNonce);
+ logAntSolutionOutcome(transaction, 0, ValidityResult::RejectNonCanonicalNonce);
+ return;
+ }
+
+ // Keep previous behavior, we fold both good and bad score into resource testing digest
+ unsigned int childAnnHash;
+ KangarooTwelve(childAnn, sizeof(*childAnn), &childAnnHash, sizeof(childAnnHash));
+ resourceTestingDigest ^= childScore;
+ resourceTestingDigest ^= childAnnHash;
+ KangarooTwelve(&resourceTestingDigest, sizeof(resourceTestingDigest), &resourceTestingDigest, sizeof(resourceTestingDigest));
+
+ const AntCommitInput in = {
+ transaction->sourcePublicKey,
+ transaction->nonce,
+ parentRef,
+ { system.tick, transactionIndex }, // selfRef, ABSOLUTE tick
+ transaction->anchorTick, // ABSOLUTE
+ system.tick }; // publishTick, ABSOLUTE (== selfRef.tick)
+ // Seeing this transaction execute, that mean those solution is recognized on-chain, mark them as recorded
+ for (unsigned int ownIdx = 0; ownIdx < computorSeedsCount; ownIdx++)
+ {
+ if (transaction->sourcePublicKey == computorPublicKeys[ownIdx])
+ {
+ gAntPendingSolutions.markRecorded(transaction->sourcePublicKey, parentRef, transaction->nonce);
+ break;
+ }
+ }
+
+ result = gAntColony.commit(in, parentRec, childScore, *childAnn, childAnnHash);
+ logAntSolutionOutcome(transaction, childScore, result);
+ antDebugAccepted(transaction, childScore, (parentRec != nullptr) ? (parentRec->depth + 1) : 1, transactionIndex, result);
+ // ValidNotStored is the store being full: the solution passed every rule and only missed a slot,
+ // so it earns its refund and its ranking exactly like a stored one
+ if (result != ValidityResult::Valid && result != ValidityResult::ValidNotStored)
+ {
+ return; // commit() counted this one itself
+ }
+
+ // Refund AND ranking. A valid solution is refunded whether or not it improved this miner's best,
+ // and whether or not the store had room for it, ranking is best-score-only
+ if (transaction->claimedScore == childScore)
+ {
+ // Refund if this score == its claimed score and the ann tree check
+ increaseEnergy(transaction->sourcePublicKey, transaction->amount);
+
+ const QuTransfer quTransfer = { m256i::zero(), transaction->sourcePublicKey, transaction->amount };
+ logger.logQuTransfer(quTransfer);
+
+ // A miner is ranked by its single best score of the epoch
+ const unsigned int newScore = childScore * gScoreMultiplier[score_engine::AlgoType::Bpp9000];
+ updateMinerRankingAndFutureComputors(transaction->sourcePublicKey, newScore, system.tick);
+ }
+}
+
static void processTickTransaction(const Transaction* transaction, unsigned int transactionIndex, unsigned long long processorNumber)
{
PROFILE_SCOPE();
@@ -2849,6 +3639,19 @@ static void processTickTransaction(const Transaction* transaction, unsigned int
}
break;
+ case AntColonyMiningSolutionTransaction::transactionType():
+ {
+ // Exact inputSize, not >=: the payload is fixed and a longer one is not a
+ // forward-compatible variant, it is a different transaction.
+ if (transaction->amount >= AntColonyMiningSolutionTransaction::minAmount()
+ && transaction->inputSize == AntColonyMiningSolutionTransaction::minInputSize())
+ {
+ processTickTransactionAntColonySolution(
+ (AntColonyMiningSolutionTransaction*)transaction, transactionIndex, processorNumber);
+ }
+ }
+ break;
+
case OracleReplyCommitTransactionPrefix::transactionType():
{
oracleEngine.processOracleReplyCommitTransaction((OracleReplyCommitTransactionPrefix*)transaction);
@@ -3131,6 +3934,89 @@ static bool makeAndBroadcastExecutionFeeTransaction(int i, BroadcastFutureTickDa
}
OPTIMIZE_OFF()
+
+// Publish at most one queued ant solution for computor i, retrying anything whose transaction never
+// landed. Mirrors the legacy solution publisher: the tick the transaction is targeted at is also the
+// deadline to see it on-chain, so missing it is what triggers the retry.
+static void publishAntSolutionFor(unsigned long long processorNumber, unsigned int computorIndex)
+{
+ AntPendingSolution entry;
+ const unsigned int idx = gAntPendingSolutions.selectForPublish(
+ computorPublicKeys[computorIndex], system.tick, entry);
+ if (idx == AntPendingSolutions::NO_ENTRY)
+ {
+ return;
+ }
+
+ const AntSolutionRecord* parentRec = nullptr;
+ if (gAntColony.tryGetParent(entry.parentRef, &parentRec) != ValidityResult::Valid)
+ {
+ gAntPendingSolutions.markObsoleteParentGone(idx);
+ antDebugPending(L"retire parentGone", entry, 0);
+ return;
+ }
+
+ m256i anchorDigest;
+ if (!gAntColony.getAnchorDigest(entry.anchorTick, anchorDigest))
+ {
+ // The ring no longer holds it, so this node could not score the transaction it is about to
+ // publish - and neither could anyone else.
+ gAntPendingSolutions.markObsoleteExpired(idx);
+ antDebugPending(L"retire expired", entry, 0);
+ return;
+ }
+
+ // Last point before the node signs with its own computor key and funds the deposit from its own
+ // balance. The commit path forfeits the deposit on a non-canonical nonce, and a check this cheap
+ // belongs on both sides of the queue.
+ if (!score_engine::ScoreEngineT::isCanonicalAntNonce(entry.nonce.m256i_u8))
+ {
+ gAntPendingSolutions.markObsoleteGateRejected(idx);
+ antDebugPending(L"retire gateRejected", entry, 0);
+ return;
+ }
+
+ // The score was computed at receipt, on a request processor, and the entry has
+ // carried it since.
+ // Judged at the tick the transaction will EXECUTE in, not the current one, so this gate sees
+ // what commit will see - a solution at the window boundary now would commit stale.
+ const unsigned int publishTick = system.tick + MIN_MINING_SOLUTIONS_PUBLICATION_OFFSET;
+ const unsigned int childCount = gAntColony.childCountForQuery(entry.parentRef,
+ entry.computorPublicKey);
+ const ChildCandidate candidate{ entry.computorPublicKey, entry.score, entry.anchorTick, publishTick };
+ if (AntColonyBpp9000T::validateChild(candidate, parentRec, childCount,
+ gAntColony.errorThreshold()) != ValidityResult::Valid)
+ {
+ gAntPendingSolutions.markObsoleteGateRejected(idx);
+ antDebugPending(L"retire gateRejected", entry, 0);
+ return;
+ }
+
+ AntColonyMiningSolutionTransaction payload;
+ setMem(&payload, sizeof(payload), 0);
+ payload.sourcePublicKey = computorPublicKeys[computorIndex];
+ payload.destinationPublicKey = m256i::zero();
+ payload.amount = AntColonyMiningSolutionTransaction::minAmount();
+ payload.tick = publishTick;
+ payload.inputType = AntColonyMiningSolutionTransaction::transactionType();
+ payload.inputSize = AntColonyMiningSolutionTransaction::minInputSize();
+ payload.parentTick = entry.parentRef.tick;
+ payload.parentSolutionIndexInTick = entry.parentRef.solutionIndexInTick;
+ payload.anchorTick = entry.anchorTick;
+ payload.claimedScore = entry.score;
+ payload.nonce = entry.nonce;
+
+ unsigned char digest[32];
+ KangarooTwelve(&payload, sizeof(Transaction) + AntColonyMiningSolutionTransaction::minInputSize(),
+ digest, sizeof(digest));
+ sign(computorSubseeds[computorIndex].m256i_u8, computorPublicKeys[computorIndex].m256i_u8,
+ digest, payload.signature);
+
+ enqueueResponse(NULL, sizeof(payload), BROADCAST_TRANSACTION, 0, &payload);
+ gAntPendingSolutions.markScheduled(idx, (int)payload.tick);
+ antDebugPending(L"published", entry, payload.tick);
+}
+
static void processTick(unsigned long long processorNumber)
{
PROFILE_SCOPE();
@@ -3208,6 +4094,7 @@ static void processTick(unsigned long long processorNumber)
ts.tickData.acquireLock();
copyMem(&nextTickData, &ts.tickData[tickIndex], sizeof(TickData));
ts.tickData.releaseLock();
+
unsigned long long solutionProcessStartTick = __rdtsc(); // for tracking the time processing solutions
if (nextTickData.epoch == system.epoch)
{
@@ -3218,6 +4105,8 @@ static void processTick(unsigned long long processorNumber)
PROFILE_NAMED_SCOPE_BEGIN("processTick(): pre-scan solutions");
// reset solution task queue
score->resetTaskQueue();
+ // Only the gate needs clearing; the score and the network are written before it is set.
+ setMem(gAntScoredReady, sizeof(gAntScoredReady), 0);
// pre-scan any solution tx and add them to solution task queue
for (unsigned int transactionIndex = 0; transactionIndex < NUMBER_OF_TRANSACTIONS_PER_TICK; transactionIndex++)
{
@@ -3247,10 +4136,33 @@ static void processTick(unsigned long long processorNumber)
if (!(minerSolutionFlags[flagIndices[0] >> 6] & (1ULL << (flagIndices[0] & 63)))
|| !(minerSolutionFlags[flagIndices[1] >> 6] & (1ULL << (flagIndices[1] & 63))))
{
- score->addTask(transaction->sourcePublicKey, solution_miningSeed, solution_nonce);
+ score->addTask(scoreLegacySolutionTask, data, sizeof(data));
}
}
}
+ // Ant solutions ride the same async queue
+ else if (isZero(transaction->destinationPublicKey)
+ && transaction->amount >= AntColonyMiningSolutionTransaction::minAmount()
+ && transaction->inputType == AntColonyMiningSolutionTransaction::transactionType()
+ && transaction->inputSize == AntColonyMiningSolutionTransaction::minInputSize())
+ {
+ const AntColonyMiningSolutionTransaction* antTx =
+ (const AntColonyMiningSolutionTransaction*)transaction;
+ AntScoreTaskPayload task;
+ task.pubkey = antTx->sourcePublicKey;
+ task.nonce = antTx->nonce;
+ task.parentRef.tick = antTx->parentTick;
+ task.parentRef.solutionIndexInTick = antTx->parentSolutionIndexInTick;
+ task.anchorTick = antTx->anchorTick;
+ task.txIdx = transactionIndex;
+ // Skip anything this node has already looked at, accepted or not
+ unsigned int antFlagIndices[2];
+ computeAntSolutionFlagIndices(task.pubkey, task.nonce, task.parentRef, antFlagIndices);
+ if (!isAntSolutionSeen(antFlagIndices))
+ {
+ score->addTask(scoreAntSolutionTask, &task, sizeof(task));
+ }
+ }
}
}
}
@@ -3258,15 +4170,10 @@ static void processTick(unsigned long long processorNumber)
PROFILE_SCOPE_END();
{
- // Process solutions in this tick and store in cache. In parallel, score->tryProcessSolution() is called by
- // request processors to speed up solution processing.
+ // Process solutions in this tick and store in cache. In parallel, request processors call
+ // score->tryProcessOneTask() from their idle path to speed this up.
PROFILE_NAMED_SCOPE("processTick(): process solutions");
- score->startProcessTaskQueue();
- while (!score->isTaskQueueProcessed())
- {
- score->tryProcessSolution(processorNumber);
- }
- score->stopProcessTaskQueue();
+ score->runUntilDone(processorNumber);
}
solutionTotalExecutionTicks = __rdtsc() - solutionProcessStartTick; // for tracking the time processing solutions
@@ -3360,6 +4267,15 @@ static void processTick(unsigned long long processorNumber)
}
}
PROFILE_SCOPE_END();
+
+ // Ant colony anchor for this non-empty tick, K12(tick || transactionDigest)
+ {
+ m256i anchorTxDigest;
+ KangarooTwelve(&nextTickData, sizeof(TickData), &anchorTxDigest, 32);
+ m256i anchorDigest;
+ computeAntAnchorDigest(system.tick, anchorTxDigest, anchorDigest);
+ gAntColony.recordAnchorDigest(system.tick, anchorDigest);
+ }
}
// Resend own oracle queries for share validation if they were scheduled for but not included in this tick.
@@ -3971,6 +4887,9 @@ static void processTick(unsigned long long processorNumber)
enqueueResponse(NULL, sizeof(payload), BROADCAST_TRANSACTION, 0, &payload);
}
+
+ // Re-publish the solutions that are not on chain
+ publishAntSolutionFor(processorNumber, i);
}
}
@@ -4063,6 +4982,7 @@ static void beginEpoch()
score->initMemory();
score->resetTaskQueue();
setMem(minerSolutionFlags, NUMBER_OF_MINER_SOLUTION_FLAGS / 8, 0);
+ setMem(gAntSolutionFlags, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, 0);
setMem((void*)minerPublicKeys, sizeof(minerPublicKeys), 0);
setMem((void*)minerScores, sizeof(minerScores), 0xFF);
setMem((void*)minerBestScoreTicks, sizeof(minerBestScoreTicks), 0);
@@ -4463,6 +5383,7 @@ static bool saveAllNodeStates()
}
score->saveScoreCache(system.epoch, directory);
+ gAntColony.saveReplayCache(system.epoch, directory);
copyMem(&nodeStateBuffer.etalonTick, &etalonTick, sizeof(etalonTick));
copyMem(nodeStateBuffer.minerPublicKeys, (void*)minerPublicKeys, sizeof(minerPublicKeys));
@@ -4548,6 +5469,14 @@ static bool saveAllNodeStates()
return false;
}
+ logToConsole(L"Saving ant solution flags");
+ savedSize = save(ANT_SOL_FLAG_FILE_NAME, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (unsigned char*)gAntSolutionFlags, directory);
+ if (savedSize != NUMBER_OF_ANT_SOLUTION_FLAGS / 8)
+ {
+ logToConsole(L"Failed to save ant solution flag");
+ return false;
+ }
+
setText(message, L"Saving tick storage ");
logToConsole(message);
if (ts.trySaveToFile(system.epoch, system.tick, directory) != 0)
@@ -4559,6 +5488,11 @@ static bool saveAllNodeStates()
#if !defined(NDEBUG)
oracleEngine.checkStateConsistencyWithAssert();
#endif
+ if (!gAntColony.saveSnapshot(system.epoch, directory, system.initialTick))
+ {
+ return false;
+ }
+
if (!oracleEngine.saveSnapshot(system.epoch, directory))
{
return false;
@@ -4807,6 +5741,32 @@ static bool loadAllNodeStates()
return false;
}
+ logToConsole(L"Loading ant solution flags");
+ loadedSize = load(ANT_SOL_FLAG_FILE_NAME, NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (unsigned char*)gAntSolutionFlags, directory);
+ if (loadedSize != NUMBER_OF_ANT_SOLUTION_FLAGS / 8)
+ {
+ logToConsole(L"Failed to load ant solution flag");
+ return false;
+ }
+
+ // initialRandomSeedFromPersistingState, not score->currentRandomSeed, the scorer is not reseeded
+ // until initialize() finishes, so this is the only restored copy available here.
+ if (!gAntColony.loadSnapshot(system.epoch, directory,
+ initialRandomSeedFromPersistingState,
+ (unsigned int)getSolutionThreshold(score_engine::AlgoType::Bpp9000),
+ system.initialTick))
+ {
+ return false;
+ }
+#ifndef NDEBUG
+ {
+ CHAR16 dbg[128];
+ setText(dbg, L"[ant-colony] snapshot loaded, solutions=");
+ appendNumber(dbg, gAntColony.solutionCount(), FALSE);
+ logToConsole(dbg);
+ }
+#endif
+
if (!oracleEngine.loadSnapshot(system.epoch, directory))
{
return false;
@@ -5838,6 +6798,16 @@ static void tickProcessor(void*)
asyncSave(REVENUE_DATA_END_OF_EPOCH_FILE_NAME, sizeof(gEpochRevenueData), (unsigned char*)&gEpochRevenueData);
// Multi-dim revenue (shadow) - for offline comparison against the additive
asyncSave(MULTIDIM_REVENUE_END_OF_EPOCH_FILE_NAME, sizeof(gMultiDimRevenue), (unsigned char*)&gMultiDimRevenue);
+ // The epoch's best networks, for offline extraction
+#ifndef NDEBUG
+ {
+ CHAR16 dbg[768];
+ setText(dbg, L"[ant-colony] epoch end: ");
+ gAntColony.stats().appendLog(dbg);
+ logToConsole(dbg);
+ }
+#endif
+ gAntColony.exportBestSolutions(system.epoch, NULL);
// Reorder futureComputors so requalifying computors keep their index
// This is needed for correct execution fee reporting across epoch boundaries
@@ -6355,11 +7325,24 @@ static bool initialize()
}
setMem(score_qpi, sizeof(*score_qpi), 0);
+ if (!gAntPendingSolutions.init())
+ {
+ return false;
+ }
+ if (!gAntColony.init())
+ {
+ return false;
+ }
+
setMem(&solutionThreshold[0][0], sizeof(int) * MAX_NUMBER_EPOCH * score_engine::AlgoType::MaxAlgoCount, 0);
if (!allocPoolWithErrorLog(L"minserSolutionFlag", NUMBER_OF_MINER_SOLUTION_FLAGS / 8, (void**)&minerSolutionFlags, __LINE__))
{
return false;
}
+ if (!allocPoolWithErrorLog(L"antSolutionFlag", NUMBER_OF_ANT_SOLUTION_FLAGS / 8, (void**)&gAntSolutionFlags, __LINE__))
+ {
+ return false;
+ }
if (!customQubicMiningStorage.init())
{
@@ -6550,16 +7533,34 @@ static bool initialize()
{
score->initMiningData(initialRandomSeedFromPersistingState);
loadMiningSeedFromFile = false;;
+ // Skipped entirely when a snapshot was restored
+ if (!loadAllNodeStateFromFile)
+ {
+ antColonyBeginEpoch();
+ }
}
else
{
- short tickEpoch = -1;
+ short tickEpoch = -1;
TimeDate tickDate;
setMem((void*)&tickDate, sizeof(TimeDate), 0);
checkAndSwitchMiningPhase(tickEpoch, tickDate, true);
- }
+ }
score->loadScoreCache(system.epoch);
+ // After the branch above, never before: both paths can call antColonyBeginEpoch(), which clears
+ // the cache. A memo, not state - absence or any load failure just means the solutions get
+ // computed honestly.
+ gAntColony.loadReplayCache(system.epoch, NULL);
+#ifndef NDEBUG
+ {
+ CHAR16 dbg[128];
+ setText(dbg, L"[ant-colony] replay cache loaded, occupancy=");
+ appendNumber(dbg, gAntColony.replayCacheOccupancy(), FALSE);
+ logToConsole(dbg);
+ }
+#endif
+
// Load + hash-verify the bpp9000 task once at init
if (!loadBpp9000Task())
{
@@ -6727,6 +7728,9 @@ static void deinitialize()
pendingTxsPool.deinit();
+ gAntPendingSolutions.deinit();
+ gAntColony.deinit();
+
if (score)
{
freePool(score);
@@ -6735,6 +7739,10 @@ static void deinitialize()
{
freePool(minerSolutionFlags);
}
+ if (gAntSolutionFlags)
+ {
+ freePool(gAntSolutionFlags);
+ }
if (dejavu0)
{
@@ -7433,6 +8441,47 @@ static void processKeyPresses()
setText(message, L"DogeMining: ");
gDogeMiningStats.appendLog(message);
logToConsole(message);
+
+ setText(message, L"AntColony: ");
+ gAntColony.stats().appendLog(message);
+ appendText(message, L" | replay cache ");
+ appendNumber(message, gAntColony.replayCacheOccupancy(), TRUE);
+ logToConsole(message);
+
+ AntPendingSolutions::Stats pending;
+ unsigned int pendingCount = 0;
+ gAntPendingSolutions.getStats(pending, pendingCount);
+ setText(message, L"AntPool: queued ");
+ appendNumber(message, pendingCount, TRUE);
+ appendText(message, L" | received ");
+ appendNumber(message, pending.received, TRUE);
+ appendText(message, L" | published ");
+ appendNumber(message, pending.published, TRUE);
+ appendText(message, L" | recorded ");
+ appendNumber(message, pending.recorded, TRUE);
+ appendText(message, L" | dropped: nonCanonical ");
+ appendNumber(message, pending.droppedNonCanonical, TRUE);
+ appendText(message, L", badAnchor ");
+ appendNumber(message, pending.droppedBadAnchor, TRUE);
+ appendText(message, L", parentUnknown ");
+ appendNumber(message, pending.droppedParentUnknown, TRUE);
+ appendText(message, L", unscorable ");
+ appendNumber(message, pending.droppedUnscorable, TRUE);
+ appendText(message, L", unacceptable ");
+ appendNumber(message, pending.droppedUnacceptable, TRUE);
+ appendText(message, L", duplicate ");
+ appendNumber(message, pending.droppedDuplicate, TRUE);
+ appendText(message, L", full ");
+ appendNumber(message, pending.droppedFull, TRUE);
+ appendText(message, L" | obsolete: parentGone ");
+ appendNumber(message, pending.obsoleteParentGone, TRUE);
+ appendText(message, L", expired ");
+ appendNumber(message, pending.obsoleteExpired, TRUE);
+ appendText(message, L", gateRejected ");
+ appendNumber(message, pending.obsoleteGateRejected, TRUE);
+ appendText(message, L" | claim mismatch ");
+ appendNumber(message, pending.claimMismatch, TRUE);
+ logToConsole(message);
}
break;
@@ -7777,6 +8826,7 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable)
#endif
unsigned long long clockTick = 0, systemDataSavingTick = 0, loggingTick = 0, peerRefreshingTick = 0, tickRequestingTick = 0;
+ unsigned long long antReplayCacheSavingTick = 0;
unsigned int tickRequestingIndicator = 0, futureTickRequestingIndicator = 0;
autoResendTickVotes.lastTick = system.initialTick;
autoResendTickVotes.lastCheck = __rdtsc();
@@ -7964,6 +9014,16 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable)
score->saveScoreCache(system.epoch);
}
#endif
+ // Deliberately outside the guard above: the cache earns its keep by being newer than
+ // the snapshot, so tying it to the snapshot's schedule would make every entry it holds
+ // one the restored tree already covers. AUX only - saving parks the solution
+ // processors for the length of the write.
+ if ((!isMainMode())
+ && curTimeTick - antReplayCacheSavingTick >= SYSTEM_DATA_SAVING_PERIOD * frequency / 1000)
+ {
+ antReplayCacheSavingTick = curTimeTick;
+ gAntColony.saveReplayCache(system.epoch, NULL);
+ }
tryResendTickVotes();
if (curTimeTick - peerRefreshingTick >= PEER_REFRESHING_PERIOD * frequency / 1000)
@@ -8290,6 +9350,7 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable)
saveSystem();
score->saveScoreCache(system.epoch);
+ gAntColony.saveReplayCache(system.epoch, NULL);
#ifdef ENABLE_PROFILING
gProfilingDataCollector.writeToFile();
#endif
@@ -8317,3 +9378,5 @@ EFI_STATUS efi_main(EFI_HANDLE imageHandle, EFI_SYSTEM_TABLE* systemTable)
return EFI_SUCCESS;
}
+
+
diff --git a/src/score.h b/src/score.h
index 99a6523ad..31f0772c1 100644
--- a/src/score.h
+++ b/src/score.h
@@ -18,20 +18,9 @@ enum ScoreStatus
ScoreStatusTaskNotLoaded,
};
-template
-struct ScoreFunction
+namespace score_engine
{
- score_engine::ScoreEngine<
- score_engine::NeuraxonParams<
- NEURAXON_NUMBER_OF_INPUT_NEURONS,
- NEURAXON_NUMBER_OF_OUTPUT_NEURONS,
- NEURAXON_NUMBER_OF_TICKS,
- NEURAXON_NUMBER_OF_NEIGHBORS,
- NEURAXON_POPULATION_THRESHOLD,
- NEURAXON_NUMBER_OF_MUTATIONS,
- NEURAXON_SOLUTION_THRESHOLD_DEFAULT>,
-
- score_engine::Bpp9000Params<
+ using Bpp9000ParamsT = Bpp9000Params<
BPP9000_NUMBER_OF_INPUT_NEURONS,
BPP9000_NUMBER_OF_OUTPUT_NEURONS,
BPP9000_SEQUENCE_LENGTH,
@@ -40,9 +29,37 @@ struct ScoreFunction
BPP9000_NUMBER_OF_NEIGHBORS,
BPP9000_POPULATION_THRESHOLD,
BPP9000_NUMBER_OF_MUTATIONS,
- BPP9000_SOLUTION_THRESHOLD_DEFAULT>
- > _computeBuffer[solutionBufferCount];
+ BPP9000_SOLUTION_THRESHOLD_DEFAULT>;
+
+ using NeuraxonParamsT = NeuraxonParams<
+ NEURAXON_NUMBER_OF_INPUT_NEURONS,
+ NEURAXON_NUMBER_OF_OUTPUT_NEURONS,
+ NEURAXON_NUMBER_OF_TICKS,
+ NEURAXON_NUMBER_OF_NEIGHBORS,
+ NEURAXON_POPULATION_THRESHOLD,
+ NEURAXON_NUMBER_OF_MUTATIONS,
+ NEURAXON_SOLUTION_THRESHOLD_DEFAULT>;
+
+ // The bpp9000 scorer the ant colony branches on; exposes ANN (the inheritable per-neuron LUT).
+ using ScoreBpp9000T = ScoreBpp9000;
+
+ using ScoreEngineT = ScoreEngine;
+}
+template
+struct ScoreFunction
+{
+private:
+ // The engine scratch buffers and the locks guarding them. Private on purpose: a work function run
+ // by the task queue cannot reach them, so it cannot take a slot lock and then call a method that
+ // takes the same one. Every route into the engine locks exactly once, inside this class.
+ score_engine::ScoreEngineT _computeBuffer[solutionBufferCount];
+ volatile char solutionEngineLock[solutionBufferCount];
+
+ // Scratch for the ant root derivation, one per engine slot and covered by that slot's own lock
+ score_engine::ScoreBpp9000T::ANN _antRootScratch[solutionBufferCount];
+
+public:
volatile char random2PoolLock;
unsigned char state[score_engine::STATE_SIZE];
unsigned char externalPoolVec[score_engine::POOL_VEC_PADDING_SIZE];
@@ -64,8 +81,6 @@ struct ScoreFunction
m256i currentRandomSeed;
- volatile char solutionEngineLock[solutionBufferCount];
-
#if USE_SCORE_CACHE
volatile char scoreCacheLock;
ScoreCache scoreCache;
@@ -81,9 +96,8 @@ struct ScoreFunction
}
currentRandomSeed = randomSeed; // persist the initial random seed to be able to send it back on system info response
- ACQUIRE(random2PoolLock);
+ LockGuard guard(random2PoolLock);
copyMem(poolVec, externalPoolVec, score_engine::POOL_VEC_PADDING_SIZE);
- RELEASE(random2PoolLock);
}
// Load the task blocks into every compute buffer; returns false if any leaf rejects them.
@@ -136,12 +150,11 @@ struct ScoreFunction
void saveScoreCache(int epoch, CHAR16* directory = NULL)
{
#if USE_SCORE_CACHE
- ACQUIRE(scoreCacheLock);
+ LockGuard guard(scoreCacheLock);
SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0';
SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0';
SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0';
scoreCache.save(SCORE_CACHE_FILE_NAME, directory);
- RELEASE(scoreCacheLock);
#endif
}
@@ -150,12 +163,13 @@ struct ScoreFunction
{
bool success = true;
#if USE_SCORE_CACHE
- ACQUIRE(scoreCacheLock);
- SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0';
- SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0';
- SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0';
- success = scoreCache.load(SCORE_CACHE_FILE_NAME);
- RELEASE(scoreCacheLock);
+ {
+ LockGuard guard(scoreCacheLock);
+ SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 4] = epoch / 100 + L'0';
+ SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 3] = (epoch % 100) / 10 + L'0';
+ SCORE_CACHE_FILE_NAME[sizeof(SCORE_CACHE_FILE_NAME) / sizeof(SCORE_CACHE_FILE_NAME[0]) - 2] = epoch % 10 + L'0';
+ success = scoreCache.load(SCORE_CACHE_FILE_NAME);
+ }
#endif
return success;
}
@@ -183,22 +197,63 @@ struct ScoreFunction
m256i getLastOutput(const unsigned long long processor_Number)
{
- ACQUIRE(solutionEngineLock[processor_Number]);
+ LockGuard guard(solutionEngineLock[processor_Number]);
+ return _computeBuffer[processor_Number].getLastOutput();
+ }
- m256i result = _computeBuffer[processor_Number].getLastOutput();
+ // Ant colony main score function
+ // score a child by inheriting its parent's network and walking it with the child's own seeds.
+ // parentAnn == nullptr means the parent is the submitter's root, which is derived here from the pubkey
+ // Returns INVALID_SCORE_VALUE for a non-canonical nonce, in which case outChildAnn is not written
+ // bestANN would still hold the previous call's network, and committing that would put one node's
+ // stale bytes into childAnnHash.
+ unsigned int computeAntChildScore(
+ const unsigned long long processor_Number,
+ const score_engine::ScoreBpp9000T::ANN* parentAnn,
+ const m256i& publicKey,
+ const m256i& nonce,
+ const m256i& anchorDigest,
+ score_engine::ScoreBpp9000T::ANN& outChildAnn)
+ {
+ const int solutionBufIdx = (int)(processor_Number % solutionBufferCount);
+ LockGuard guard(solutionEngineLock[solutionBufIdx]);
+ score_engine::ScoreBpp9000T& engine = _computeBuffer[solutionBufIdx]._bpp9000Score;
+
+ // Derived into this slot's scratch rather than the engine's own buffer: deriveRootANN() uses
+ // currentANN as working space
+ const score_engine::ScoreBpp9000T::ANN* parent = parentAnn;
+ // Depth 1, the start node of every public key
+ if (parent == nullptr)
+ {
+ engine.deriveRootANN(publicKey.m256i_u8, poolVec, _antRootScratch[solutionBufIdx]);
+ parent = &_antRootScratch[solutionBufIdx];
+ }
- RELEASE(solutionEngineLock[processor_Number]);
- return result;
+ const unsigned int childScore = engine.computeScoreFromParent(
+ *parent, publicKey.m256i_u8, nonce.m256i_u8, anchorDigest.m256i_u8, poolVec);
+ if (childScore == score_engine::INVALID_SCORE_VALUE)
+ {
+ return childScore;
+ }
+ engine.getBestANN(outChildAnn);
+ return childScore;
}
// main score function
unsigned int operator()(const unsigned long long processor_Number, const m256i& publicKey, const m256i& miningSeed, const m256i& nonce)
{
PROFILE_SCOPE();
- // TODO: When neuraxon's going, this check need to be modified
- if (!score_engine::isCanonicalBpp9000Nonce(nonce.m256i_u8))
+ switch (score_engine::getAlgoType(nonce.m256i_u8))
{
- return score_engine::INVALID_SCORE_VALUE;
+ case score_engine::AlgoType::Bpp9000:
+ if (!score_engine::isCanonicalBpp9000Nonce(nonce.m256i_u8))
+ {
+ return score_engine::INVALID_SCORE_VALUE;
+ }
+ break;
+ default:
+ // Unsupported algo
+ return score_engine::INVALID_SCORE_VALUE;
}
if (isZero(miningSeed) || miningSeed != currentRandomSeed)
@@ -218,11 +273,11 @@ struct ScoreFunction
#endif
const int solutionBufIdx = (int)(processor_Number % solutionBufferCount);
- ACQUIRE(solutionEngineLock[solutionBufIdx]);
-
- score = computeScore(solutionBufIdx, publicKey, nonce);
-
- RELEASE(solutionEngineLock[solutionBufIdx]);
+ {
+ // Scoped so the cache write below happens with the engine slot released.
+ LockGuard guard(solutionEngineLock[solutionBufIdx]);
+ score = computeScore(solutionBufIdx, publicKey, nonce);
+ }
#if USE_SCORE_CACHE
scoreCache.addEntry(publicKey, miningSeed, nonce, scoreCacheIndex, score);
#endif
@@ -237,107 +292,144 @@ struct ScoreFunction
unsigned long long stackSize = 0;
#endif
- // Multithreaded solutions verification:
- // This module mainly serve tick processor in qubic core node, thus the queue size is limited at NUMBER_OF_TRANSACTIONS_PER_TICK
- // for future use for somewhere else, you can only increase the size.
+ // Multithreaded solutions verification.
+ //
+ // A task is a (work function, payload) pair rather than a fixed tuple, so different kinds of
+ // scoring work can share one queue and one drain: the queue arbitrates nothing except who runs
+ // next. The payload is COPIED in, so the caller may reuse or discard its buffer immediately - a
+ // pointer here would make every caller responsible for keeping data alive across a drain.
+ //
+ // The work function is responsible for taking whatever locks it needs, including
+ // solutionEngineLock. The queue must not take it: solutionEngineLock is a non-reentrant spinlock
+ // and operator() takes it itself, so a queue that pre-acquired would deadlock any work function
+ // that reuses operator().
+ typedef void (*WorkFunc)(unsigned long long processorNumber, void* payload);
+
+ static constexpr unsigned int TASK_PAYLOAD_MAX = 128;
+
+private:
+ static constexpr unsigned int TASK_QUEUE_CAPACITY = NUMBER_OF_TRANSACTIONS_PER_TICK;
+
+ struct Task
+ {
+ WorkFunc func;
+ // 8-byte aligned: m256i is a plain union accessed with unaligned intrinsics, so it needs no more.
+ unsigned long long payload[TASK_PAYLOAD_MAX / sizeof(unsigned long long)];
+ };
volatile char taskQueueLock = 0;
- struct
- {
- m256i publicKey[NUMBER_OF_TRANSACTIONS_PER_TICK];
- m256i miningSeed[NUMBER_OF_TRANSACTIONS_PER_TICK];
- m256i nonce[NUMBER_OF_TRANSACTIONS_PER_TICK];
- } taskQueue;
+ Task taskQueue[TASK_QUEUE_CAPACITY];
unsigned int _nTask;
unsigned int _nProcessing;
unsigned int _nFinished;
- bool _nIsTaskQueueReady;
+ volatile bool _nIsTaskQueueReady;
+public:
void resetTaskQueue()
{
- ACQUIRE(taskQueueLock);
+ LockGuard guard(taskQueueLock);
_nTask = 0;
_nProcessing = 0;
_nFinished = 0;
_nIsTaskQueueReady = false;
- RELEASE(taskQueueLock);
}
- // add task to the queue
- // queue size is limited at NUMBER_OF_TRANSACTIONS_PER_TICK
- void addTask(m256i publicKey, m256i miningSeed, m256i nonce)
+ // Copies size bytes of data. Returns false if the queue is full or the payload does not fit.
+ bool addTask(WorkFunc func, const void* data, unsigned int size)
{
- ACQUIRE(taskQueueLock);
- if (_nTask < NUMBER_OF_TRANSACTIONS_PER_TICK)
+ if (size > TASK_PAYLOAD_MAX)
{
- unsigned int index = _nTask++;
- taskQueue.publicKey[index] = publicKey;
- taskQueue.miningSeed[index] = miningSeed;
- taskQueue.nonce[index] = nonce;
+ return false;
}
- RELEASE(taskQueueLock);
- }
- void startProcessTaskQueue()
- {
- ACQUIRE(taskQueueLock);
- _nIsTaskQueueReady = true;
- RELEASE(taskQueueLock);
+ LockGuard guard(taskQueueLock);
+ if (_nTask >= TASK_QUEUE_CAPACITY)
+ {
+ return false;
+ }
+ Task& t = taskQueue[_nTask++];
+ t.func = func;
+ copyMem(t.payload, data, size);
+ return true;
}
- void stopProcessTaskQueue()
+ // Outcome of one dispatch attempt, so a caller waiting for the batch does not need a second
+ // lock acquisition just to ask whether it is over.
+ enum TaskDispatchResult
{
- ACQUIRE(taskQueueLock);
- _nIsTaskQueueReady = false;
- RELEASE(taskQueueLock);
- }
+ TaskRan, // a task was taken and executed
+ TaskNonePending, // nothing left to take, but tasks are still running elsewhere
+ TaskAllDone // every queued task has finished
+ };
- // get a task, can call on any thread
- bool getTask(m256i* publicKey, m256i* miningSeed, m256i* nonce)
+ // Run one task if any is pending. Called from request processors' idle path and from the drain.
+ TaskDispatchResult tryProcessOneTask(unsigned long long processorNumber)
{
if (!_nIsTaskQueueReady)
{
- return false;
+ // No thing to process
+ return TaskNonePending;
}
- bool result = false;
- ACQUIRE(taskQueueLock);
- if (_nProcessing < _nTask)
+
+ WorkFunc func = nullptr;
+ unsigned long long payload[TASK_PAYLOAD_MAX / sizeof(unsigned long long)];
+ TaskDispatchResult result = TaskNonePending;
+
+ // The task itself must run with the lock released
{
- unsigned int index = _nProcessing++;
- *publicKey = taskQueue.publicKey[index];
- *miningSeed = taskQueue.miningSeed[index];
- *nonce = taskQueue.nonce[index];
- result = true;
+ LockGuard guard(taskQueueLock);
+ if (_nFinished >= _nTask)
+ {
+ result = TaskAllDone;
+ }
+ else if (_nIsTaskQueueReady && _nProcessing < _nTask)
+ {
+ const Task& t = taskQueue[_nProcessing++];
+ func = t.func;
+ copyMem(payload, t.payload, TASK_PAYLOAD_MAX);
+ result = TaskRan;
+ }
}
- else
+
+ if (func == nullptr)
{
- result = false;
+ return result;
+ }
+ func(processorNumber, payload);
+
+ {
+ LockGuard guard(taskQueueLock);
+ _nFinished++;
}
- RELEASE(taskQueueLock);
return result;
}
- void finishTask()
- {
- ACQUIRE(taskQueueLock);
- _nFinished++;
- RELEASE(taskQueueLock);
- }
- bool isTaskQueueProcessed()
+ // Open the queue and work it down. The caller participates rather than spinning idle, and returns
+ // only once every task has finished - including those running on other threads
+ void runUntilDone(unsigned long long processorNumber)
{
- return _nFinished == _nTask;
- }
+ {
+ LockGuard guard(taskQueueLock);
+ _nIsTaskQueueReady = true;
+ }
+
+ // Wait for task queue finish
+ for (;;)
+ {
+ const TaskDispatchResult result = tryProcessOneTask(processorNumber);
+ if (result == TaskAllDone)
+ {
+ break;
+ }
+ if (result == TaskNonePending)
+ {
+ _mm_pause();
+ }
+ }
- void tryProcessSolution(unsigned long long processorNumber)
- {
- m256i publicKey;
- m256i miningSeed;
- m256i nonce;
- bool res = this->getTask(&publicKey, &miningSeed, &nonce);
- if (res)
{
- (*this)(processorNumber, publicKey, miningSeed, nonce);
- this->finishTask();
+ LockGuard guard(taskQueueLock);
+ _nIsTaskQueueReady = false;
}
}
};
diff --git a/test/ant_colony.cpp b/test/ant_colony.cpp
new file mode 100644
index 000000000..abda9c9bf
--- /dev/null
+++ b/test/ant_colony.cpp
@@ -0,0 +1,993 @@
+#define NO_UEFI
+
+#include "gtest/gtest.h"
+
+#define ENABLE_PROFILING 0
+
+// The bound colony, not the bare template: these tests check bpp9000's binding as well as the rules.
+#include "../src/mining/ant_colony/ant_colony_bpp9000.h"
+
+#include
+
+static constexpr unsigned int TEST_THRESHOLD = 3838; // BPP9000_SOLUTION_THRESHOLD_DEFAULT
+// Ticks are absolute. A commit at TEST_PUBLISH_TICK lands in tick-index slot (TEST_PUBLISH_TICK -
+// TEST_INITIAL_TICK), which must stay under MAX_NUMBER_OF_TICKS_PER_EPOCH (3005 on the testnet setting).
+static constexpr unsigned int TEST_INITIAL_TICK = 99000;
+static constexpr unsigned int TEST_PUBLISH_TICK = 100000;
+
+static m256i makeKey(unsigned long long n)
+{
+ m256i k = m256i::zero();
+ k.m256i_u64[0] = n + 1;
+ return k;
+}
+
+// A parent sitting at the given score, owned by the given identity.
+static AntSolutionRecord makeParent(const m256i& owner, unsigned int score, unsigned int depth = 1)
+{
+ AntSolutionRecord r;
+ setMem(&r, sizeof(r), 0);
+ r.pubkey = owner;
+ r.score = score;
+ r.depth = depth;
+ r.parentRef = ROOT_REF;
+ r.nextSiblingIdx = NO_SIBLING;
+ return r;
+}
+
+// A candidate from `owner` at `score`, anchored and published in the same tick unless the test is
+// about freshness.
+static ChildCandidate makeChild(const m256i& owner, unsigned int score,
+ unsigned int anchorTick = 1000, unsigned int publishTick = 1000)
+{
+ ChildCandidate c;
+ c.pubkey = owner;
+ c.score = score;
+ c.anchorTick = anchorTick;
+ c.publishTick = publishTick;
+ return c;
+}
+
+// Every test runs at TEST_THRESHOLD, so wrapping it keeps the assertions on one line.
+static ValidityResult admit(const ChildCandidate& child, const AntSolutionRecord* parent,
+ unsigned int childCount)
+{
+ return AntColonyBpp9000T::validateChild(child, parent, childCount, TEST_THRESHOLD);
+}
+
+// The packing itself is generic and tested exhaustively
+TEST(TestAntColonyPackedAnn, CoversAWholeAnnAtTheUnpaddedStride)
+{
+ AntColonyBpp9000T::Ann src;
+ for (unsigned long long i = 0; i < sizeof(src); i++)
+ {
+ src.lut[i] = (unsigned char)(i % 3); // mutate() only ever writes 0, 1 or 2
+ }
+
+ AntColonyBpp9000T::PackedAnn packed;
+ packed.pack(src.lut);
+
+ AntColonyBpp9000T::Ann back;
+ setMem(&back, sizeof(back), 0xFF);
+ packed.unpack(back.lut);
+
+ for (unsigned long long i = 0; i < sizeof(src); i++)
+ {
+ ASSERT_EQ(back.lut[i], src.lut[i]) << "entry " << i;
+ }
+}
+
+// The threshold is checked before the parent comparison, so nodes worse than it are never stored
+TEST(TestAntColonyValidate, ThresholdIsAnUpperBoundOnError)
+{
+ const m256i me = makeKey(1);
+
+ // A parent that would otherwise admit anything, so only the threshold can reject.
+ const AntSolutionRecord looseParent = makeParent(me, WORST_SCORE);
+ EXPECT_EQ(admit(makeChild(me, 3839), &looseParent, 0),
+ ValidityResult::RejectBelowThreshold);
+
+ // Exactly at the bound is accepted: the rule is score > threshold, not >=.
+ EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD), &looseParent, 0), ValidityResult::Valid);
+}
+
+TEST(TestAntColonyValidate, MustStrictlyBeatParent)
+{
+ const m256i me = makeKey(2);
+ const AntSolutionRecord parent = makeParent(me, 3800);
+
+ EXPECT_EQ(admit(makeChild(me, 3799), &parent, 0), ValidityResult::Valid);
+ EXPECT_EQ(admit(makeChild(me, 3800), &parent, 0), ValidityResult::RejectLeParent);
+ EXPECT_EQ(admit(makeChild(me, 3801), &parent, 0), ValidityResult::RejectLeParent);
+}
+
+// A root has no score of its own, so any threshold-passing child improves on it. This is what lets a
+// lineage start at all.
+TEST(TestAntColonyValidate, RootParentAdmitsAnyPassingScore)
+{
+ const m256i me = makeKey(3);
+
+ EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD), nullptr, 0), ValidityResult::Valid);
+ EXPECT_EQ(admit(makeChild(me, 0), nullptr, 0), ValidityResult::Valid);
+ EXPECT_EQ(admit(makeChild(me, TEST_THRESHOLD + 1), nullptr, 0),
+ ValidityResult::RejectBelowThreshold);
+}
+
+// Trees are isolated per identity: a miner cannot branch off someone else's node.
+TEST(TestAntColonyValidate, CannotBranchFromAnotherIdentity)
+{
+ const m256i me = makeKey(4);
+ const m256i someoneElse = makeKey(5);
+ const AntSolutionRecord theirNode = makeParent(someoneElse, 3800);
+
+ EXPECT_EQ(admit(makeChild(me, 3700), &theirNode, 0), ValidityResult::RejectWrongTree);
+
+ const AntSolutionRecord myNode = makeParent(me, 3800);
+ EXPECT_EQ(admit(makeChild(me, 3700), &myNode, 0), ValidityResult::Valid);
+}
+
+// Per-parent child cap. The cap is compile-time; 0 means unbound.
+TEST(TestAntColonyValidate, RejectsAtTheChildCap)
+{
+ const m256i me = makeKey(6);
+ const AntSolutionRecord parent = makeParent(me, WORST_SCORE);
+
+ // Below the cap - and always, when unbound - a passing child is admitted.
+ EXPECT_EQ(admit(makeChild(me, 3799), &parent, 0), ValidityResult::Valid);
+
+ // At the cap it is refused. Skipped when unbound (0). The runtime copy keeps the compile-time
+ // zero from tripping a constant-condition warning.
+ const unsigned int cap = ANT_MAX_CHILDREN_PER_PARENT;
+ if (cap != 0)
+ {
+ EXPECT_EQ(admit(makeChild(me, 3799), &parent, cap),
+ ValidityResult::RejectMaxChildrenPerParent);
+ }
+}
+
+// Freshness
+TEST(TestAntColonyValidate, FreshnessWindowBoundaries)
+{
+ const m256i me = makeKey(7);
+ const AntSolutionRecord parent = makeParent(me, WORST_SCORE);
+ const unsigned int anchor = 100000;
+
+ // Published in the same tick it anchored to: the tightest legal case.
+ EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor), &parent, 0),
+ ValidityResult::Valid);
+
+ // Exactly at the window edge is still legal; one past it is not.
+ EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor + ANT_PUBLISH_WINDOW_TICKS),
+ &parent, 0), ValidityResult::Valid);
+ EXPECT_EQ(admit(makeChild(me, 3700, anchor, anchor + ANT_PUBLISH_WINDOW_TICKS + 1),
+ &parent, 0), ValidityResult::RejectStale);
+
+ // An anchor in the future is rejected rather than wrapping the unsigned subtraction.
+ EXPECT_EQ(admit(makeChild(me, 3700, anchor + 1, anchor), &parent, 0),
+ ValidityResult::RejectStale);
+}
+
+// Order of checks: Freshness first, then tree isolation, then threshold, then
+// parent, then the child cap.
+TEST(TestAntColonyValidate, ReportsTheFirstFailingRule)
+{
+ const m256i me = makeKey(8);
+ const m256i other = makeKey(9);
+ const unsigned int anchor = 100000;
+ const unsigned int stalePublish = anchor + ANT_PUBLISH_WINDOW_TICKS + 1;
+
+ const AntSolutionRecord theirs = makeParent(other, 3000);
+
+ // Stale AND wrong tree AND above threshold AND worse than parent -> reports Stale.
+ EXPECT_EQ(admit(makeChild(me, 9999, anchor, stalePublish), &theirs, 0),
+ ValidityResult::RejectStale);
+
+ // Fresh, but wrong tree AND above threshold -> reports WrongTree.
+ EXPECT_EQ(admit(makeChild(me, 9999, anchor, anchor), &theirs, 0),
+ ValidityResult::RejectWrongTree);
+
+ // Own tree, above threshold AND worse than parent -> reports the threshold.
+ const AntSolutionRecord mine = makeParent(me, 3000);
+ EXPECT_EQ(admit(makeChild(me, 9999, anchor, anchor), &mine, 0),
+ ValidityResult::RejectBelowThreshold);
+
+ // Passes the threshold but worse than parent -> reports the parent.
+ EXPECT_EQ(admit(makeChild(me, 3500, anchor, anchor), &mine, 0),
+ ValidityResult::RejectLeParent);
+}
+
+// For a fixed parent and threshold, acceptance
+// must be monotone in the score - every score at or below the tightest bound is accepted, every
+// score above it is rejected. An inverted comparison anywhere breaks this even if the individual
+// boundary tests above were adjusted to match it.
+TEST(TestAntColonyValidate, AcceptanceIsMonotoneInScore)
+{
+ const m256i me = makeKey(10);
+ const unsigned int parentScore = 3800;
+ const AntSolutionRecord parent = makeParent(me, parentScore);
+
+ // Tightest of: <= threshold, < parent.
+ const unsigned int bestRejected = parentScore;
+
+ bool sawAccept = false;
+ for (unsigned int score = 3700; score <= 3900; score++)
+ {
+ const ValidityResult r = admit(makeChild(me, score), &parent, 0);
+ const bool accepted = (r == ValidityResult::Valid);
+ if (score < bestRejected && score <= TEST_THRESHOLD)
+ {
+ ASSERT_TRUE(accepted) << "score " << score << " should be accepted, got " << (int)r;
+ sawAccept = true;
+ }
+ else
+ {
+ ASSERT_FALSE(accepted) << "score " << score << " should be rejected";
+ }
+ }
+ EXPECT_TRUE(sawAccept) << "the sweep must cover the accepting region";
+}
+
+// init() allocates ~6.2 GB, so the colony is built once for the file and re-seeded between tests.
+// Lazy rather than SetUpTestSuite: that does not exist before gtest 1.10, and test.vcxproj builds
+// against 1.8.1, where it would compile clean and never run.
+static AntColonyBpp9000T* freshColony()
+{
+ static AntColonyBpp9000T colony;
+ static bool allocated = false;
+ static bool allocationFailed = false;
+
+ if (!allocated && !allocationFailed)
+ {
+ allocationFailed = !colony.init();
+ allocated = !allocationFailed;
+ }
+ if (allocationFailed)
+ {
+ return nullptr;
+ }
+
+ colony.beginEpoch(makeKey(999), TEST_INITIAL_TICK);
+ colony.setErrorThreshold(TEST_THRESHOLD);
+ return &colony;
+}
+
+// Commits one child of the root, returns its index or ANT_INVALID_INDEX. nonceSeed keeps calls
+// distinct, since (pubkey, nonce, parentRef) is the replay key.
+static long long commitRootChild(AntColonyBpp9000T* colony, const m256i& owner, unsigned int score,
+ unsigned int txIdx, unsigned long long nonceSeed, unsigned int tick = 100000)
+{
+ AntCommitInput in;
+ in.pubkey = owner;
+ in.nonce = makeKey(nonceSeed);
+ in.parentRef = ROOT_REF;
+ in.selfRef.tick = tick;
+ in.selfRef.solutionIndexInTick = txIdx;
+ in.anchorTick = tick;
+ in.publishTick = tick;
+
+ AntColonyBpp9000T::Ann ann;
+ setMem(&ann, sizeof(ann), 0);
+ ann.lut[0] = (unsigned char)(score % 3);
+
+ // The real hash, not a stand-in: the snapshot rebuild re-derives it from the stored network.
+ unsigned int annHash;
+ KangarooTwelve(&ann, sizeof(ann), &annHash, sizeof(annHash));
+
+ const long long landsAt = (long long)colony->solutionCount();
+ if (colony->commit(in, nullptr, score, ann, annHash) != ValidityResult::Valid)
+ {
+ return ANT_INVALID_INDEX;
+ }
+ return landsAt;
+}
+
+// A child of an existing node, so a test can build a lineage rather than a flat set of root children.
+static long long commitChild(AntColonyBpp9000T* colony, const m256i& owner, const SolutionRef& parentRef,
+ unsigned int score, unsigned int txIdx, unsigned long long nonceSeed, unsigned int tick = 100000)
+{
+ const AntSolutionRecord* parentRec = nullptr;
+ if (colony->tryGetParent(parentRef, &parentRec) != ValidityResult::Valid)
+ {
+ return ANT_INVALID_INDEX;
+ }
+
+ AntCommitInput in;
+ in.pubkey = owner;
+ in.nonce = makeKey(nonceSeed);
+ in.parentRef = parentRef;
+ in.selfRef.tick = tick;
+ in.selfRef.solutionIndexInTick = txIdx;
+ in.anchorTick = tick;
+ in.publishTick = tick;
+
+ AntColonyBpp9000T::Ann ann;
+ setMem(&ann, sizeof(ann), 0);
+ ann.lut[0] = (unsigned char)(score % 3);
+ unsigned int annHash;
+ KangarooTwelve(&ann, sizeof(ann), &annHash, sizeof(annHash));
+
+ const long long landsAt = (long long)colony->solutionCount();
+ if (colony->commit(in, parentRec, score, ann, annHash) != ValidityResult::Valid)
+ {
+ return ANT_INVALID_INDEX;
+ }
+ return landsAt;
+}
+
+// commit() head-inserts, so children chain from newest to oldest. countChildren() walks this chain
+// from the head, so it must stay intact and terminate.
+TEST(TestAntColonyStore, SiblingsChainNewestFirst)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(1);
+ // Same anchor tick, so all three are inside the freshness window and coexist.
+ const long long a = commitRootChild(colony, me, 3800, 0, 500);
+ const long long b = commitRootChild(colony, me, 3810, 1, 501);
+ const long long c = commitRootChild(colony, me, 3820, 2, 502);
+ ASSERT_NE(a, ANT_INVALID_INDEX);
+ ASSERT_NE(b, ANT_INVALID_INDEX);
+ ASSERT_NE(c, ANT_INVALID_INDEX);
+
+ EXPECT_EQ(colony->recordAt(c)->nextSiblingIdx, (unsigned int)b);
+ EXPECT_EQ(colony->recordAt(b)->nextSiblingIdx, (unsigned int)a);
+ EXPECT_EQ(colony->recordAt(a)->nextSiblingIdx, NO_SIBLING);
+}
+
+// parentRef is a logical address, so it must map back to the record index.
+TEST(TestAntColonyStore, SolutionRefResolvesToItsRecord)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(2);
+ const long long idx = commitRootChild(colony, me, 3800, 42, 600);
+ ASSERT_NE(idx, ANT_INVALID_INDEX);
+
+ const SolutionRef ref = { TEST_PUBLISH_TICK,42 };
+ EXPECT_EQ(colony->findIndexBySolutionRef(ref), idx);
+
+ // An uncommitted ref must not resolve to a neighbour.
+ const SolutionRef missing = { TEST_PUBLISH_TICK,43 };
+ EXPECT_EQ(colony->findIndexBySolutionRef(missing), ANT_INVALID_INDEX);
+}
+
+// Same (pubkey, nonce, parentRef) is a replay whatever its score.
+TEST(TestAntColonyStore, SameSolutionCannotCommitTwice)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(3);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 700), ANT_INVALID_INDEX);
+
+ EXPECT_EQ(commitRootChild(colony, me, 3700, 1, 700), ANT_INVALID_INDEX);
+ EXPECT_EQ(colony->stats().rejectReplay, 1u);
+ EXPECT_EQ(colony->solutionCount(), 1u);
+}
+
+// ROOT is never stored, so resolving it is Valid with a null record, not a lookup failure.
+TEST(TestAntColonyStore, RootRefResolvesToValidWithNoRecord)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const AntSolutionRecord* parent = (const AntSolutionRecord*)1; // must be overwritten
+ EXPECT_EQ(colony->tryGetParent(ROOT_REF, &parent), ValidityResult::Valid);
+ EXPECT_EQ(parent, nullptr);
+
+ const SolutionRef missing = { TEST_PUBLISH_TICK,0 };
+ EXPECT_EQ(colony->tryGetParent(missing, &parent), ValidityResult::RejectParentNotRegistered);
+}
+
+// Anchor ring
+TEST(TestAntColonyStore, AnchorDigestRoundTrips)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i digest = makeKey(4242);
+ colony->recordAnchorDigest(100000, digest);
+
+ m256i out = m256i::zero();
+ EXPECT_TRUE(colony->getAnchorDigest(100000, out));
+ EXPECT_TRUE(out == digest);
+
+ // An unrecorded tick is a miss, not whatever sits in that slot.
+ EXPECT_FALSE(colony->getAnchorDigest(100001, out));
+}
+
+// A tick ANT_ANCHOR_RING_SIZE later lands in the same slot. The evicted one must miss - returning
+// the new digest would score against a network the miner never used.
+TEST(TestAntColonyStore, AgedOutAnchorIsAMissNotTheWrongDigest)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const unsigned int oldTick = 100000;
+ const unsigned int newTick = oldTick + ANT_ANCHOR_RING_SIZE;
+ const m256i oldDigest = makeKey(11);
+ const m256i newDigest = makeKey(22);
+
+ colony->recordAnchorDigest(oldTick, oldDigest);
+ colony->recordAnchorDigest(newTick, newDigest);
+
+ m256i out = m256i::zero();
+ EXPECT_FALSE(colony->getAnchorDigest(oldTick, out));
+ EXPECT_TRUE(colony->getAnchorDigest(newTick, out));
+ EXPECT_TRUE(out == newDigest);
+}
+
+// beginEpoch() wipes the ring. It fills with ANT_ANCHOR_TICK_NONE, not zero, so tick 0 does not
+// look recorded.
+TEST(TestAntColonyStore, EpochResetClearsTheRing)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ colony->recordAnchorDigest(100000, makeKey(7));
+ m256i out = m256i::zero();
+ ASSERT_TRUE(colony->getAnchorDigest(100000, out));
+
+ colony->beginEpoch(makeKey(999), TEST_INITIAL_TICK);
+ EXPECT_FALSE(colony->getAnchorDigest(100000, out));
+ EXPECT_FALSE(colony->getAnchorDigest(0, out));
+}
+
+
+// Snapshot test cases
+
+static constexpr unsigned short TEST_EPOCH = 200;
+static const m256i TEST_ROOT_SEED = makeKey(999); // what freshColony() seeds with
+
+// Save, wipe, load. beginEpoch() clears everything the load has to bring back, so anything that
+// survives came out of the files.
+static bool saveWipeLoad(AntColonyBpp9000T* colony)
+{
+ if (!colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK))
+ {
+ return false;
+ }
+ colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK);
+ colony->setErrorThreshold(TEST_THRESHOLD);
+ return colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK);
+}
+
+// Records and the tick index come back, and the sibling chain is rebuilt to the same shape commit()
+// built. Only the records are on disk - nextSiblingIdx is replayed, so matching the pre-save chain
+// is what proves the replay reproduces the head-insert.
+TEST(TestAntColonySnapshot, RoundTripRestoresTheTree)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(1);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 500), ANT_INVALID_INDEX);
+ ASSERT_NE(commitRootChild(colony, me, 3810, 1, 501), ANT_INVALID_INDEX);
+ ASSERT_NE(commitRootChild(colony, me, 3820, 2, 502), ANT_INVALID_INDEX);
+
+ ASSERT_TRUE(saveWipeLoad(colony));
+
+ ASSERT_EQ(colony->solutionCount(), 3u);
+ EXPECT_EQ(colony->recordAt(2)->nextSiblingIdx, 1u);
+ EXPECT_EQ(colony->recordAt(1)->nextSiblingIdx, 0u);
+ EXPECT_EQ(colony->recordAt(0)->nextSiblingIdx, NO_SIBLING);
+ EXPECT_EQ(colony->recordAt(1)->score, 3810u);
+ EXPECT_TRUE(colony->recordAt(1)->pubkey == me);
+
+ // The tick index is derived too, so resolving a logical ref proves it was rebuilt.
+ const SolutionRef ref = { TEST_PUBLISH_TICK,1 };
+ EXPECT_EQ(colony->findIndexBySolutionRef(ref), 1LL);
+}
+
+// The stored network must come back byte for byte, otherwise children score against a parent the
+// rest of the network does not have.
+TEST(TestAntColonySnapshot, RoundTripRestoresTheStoredNetwork)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(2);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 900), ANT_INVALID_INDEX);
+
+ AntColonyBpp9000T::Ann before;
+ ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(0), before));
+
+ ASSERT_TRUE(saveWipeLoad(colony));
+
+ AntColonyBpp9000T::Ann after;
+ ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(0), after));
+ for (unsigned long long i = 0; i < sizeof(before); i++)
+ {
+ ASSERT_EQ(after.lut[i], before.lut[i]) << "entry " << i;
+ }
+}
+
+// The dedup set is not written to disk. If the rebuild misses it, a restarted node re-accepts
+// solutions it already has.
+TEST(TestAntColonySnapshot, DedupIsRebuiltSoReplaysStillFail)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(3);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 600), ANT_INVALID_INDEX);
+ ASSERT_TRUE(saveWipeLoad(colony));
+
+ // Same (pubkey, nonce, parentRef) as before the restart.
+ EXPECT_EQ(commitRootChild(colony, me, 3700, 1, 600), ANT_INVALID_INDEX);
+ EXPECT_EQ(colony->solutionCount(), 1u);
+}
+
+// A cold ring would reject solutions anchored before the restart that peers accept.
+TEST(TestAntColonySnapshot, AnchorRingSurvivesTheRoundTrip)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i digest = makeKey(4242);
+ colony->recordAnchorDigest(100000, digest);
+ ASSERT_TRUE(saveWipeLoad(colony));
+
+ m256i out = m256i::zero();
+ EXPECT_TRUE(colony->getAnchorDigest(100000, out));
+ EXPECT_TRUE(out == digest);
+ EXPECT_FALSE(colony->getAnchorDigest(100001, out));
+}
+
+// The seed and threshold are supplied by the node, not read from the file. A disagreement means the
+// colony files and the node state are from different moments, so the tree is refused.
+TEST(TestAntColonySnapshot, FileMustAgreeWithTheNodeState)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ ASSERT_NE(commitRootChild(colony, makeKey(4), 3800, 0, 700), ANT_INVALID_INDEX);
+ ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK));
+
+ EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, makeKey(12345), TEST_THRESHOLD, TEST_INITIAL_TICK));
+ EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD + 1, TEST_INITIAL_TICK));
+
+ // A different base would resolve every parentRef to the wrong record.
+ EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK + 1));
+
+ // The epoch is part of the file name, so a different one finds no snapshot at all.
+ EXPECT_FALSE(colony->loadSnapshot((unsigned short)(TEST_EPOCH + 1), NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK));
+
+ // And the matching one still loads, so the refusals above were the checks and not a bad file.
+ EXPECT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK));
+}
+
+// Those refusals all happen while only the meta has been read, so the tree the node is already
+// running on must be left alone.
+TEST(TestAntColonySnapshot, RefusedLoadLeavesTheRunningTreeIntact)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ const m256i me = makeKey(5);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 800), ANT_INVALID_INDEX);
+ ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK));
+ ASSERT_NE(commitRootChild(colony, me, 3790, 1, 801), ANT_INVALID_INDEX);
+ ASSERT_EQ(colony->solutionCount(), 2u);
+
+ EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, makeKey(12345), TEST_THRESHOLD, TEST_INITIAL_TICK));
+ EXPECT_EQ(colony->solutionCount(), 2u);
+}
+
+// An empty colony still writes all three files, so an operator's backup is always the same set and a
+// short one means a lost file rather than an empty epoch.
+TEST(TestAntColonySnapshot, EmptyColonyWritesTheFullFileSet)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ ASSERT_EQ(colony->solutionCount(), 0u);
+
+ // Clear whatever an earlier test left at this epoch, so the files below can only come from the
+ // save under test.
+ antSnapshotNameForEpoch(TEST_EPOCH);
+ _wremove(ANT_SNAPSHOT_HEADER_FILENAME);
+ _wremove(ANT_SNAPSHOT_RECORDS_FILENAME);
+ _wremove(ANT_SNAPSHOT_POOL_FILENAME);
+
+ ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK));
+
+ // All three are written. Records and pool hold a full slot even when empty, so neither is zero
+ // length; the header always carries the meta, anchor ring and export set.
+ AntColonySnapshotMeta metaSlot;
+ AntSolutionRecord recordSlot;
+ AntColonyBpp9000T::PackedAnn poolSlot;
+ EXPECT_EQ(load(ANT_SNAPSHOT_HEADER_FILENAME, sizeof(metaSlot), (unsigned char*)&metaSlot),
+ (long long)sizeof(metaSlot));
+ EXPECT_EQ(load(ANT_SNAPSHOT_RECORDS_FILENAME, sizeof(recordSlot), (unsigned char*)&recordSlot),
+ (long long)sizeof(recordSlot));
+ EXPECT_EQ(load(ANT_SNAPSHOT_POOL_FILENAME, sizeof(poolSlot), (unsigned char*)&poolSlot),
+ (long long)sizeof(poolSlot));
+
+ EXPECT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK));
+ EXPECT_EQ(colony->solutionCount(), 0u);
+}
+
+// childAnnHash is the only thing tying a record to its stored network, so it is the only check on
+// the pool file. Overwrite the pool behind the colony's back and the load must refuse.
+TEST(TestAntColonySnapshot, CorruptedPoolIsRefused)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.2 GB";
+
+ // score % 3 == 2, so an all-zero network is not the one this record hashes to.
+ ASSERT_NE(commitRootChild(colony, makeKey(6), 3800, 0, 1000), ANT_INVALID_INDEX);
+ ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK));
+
+ AntColonyBpp9000T::PackedAnn junk;
+ setMem(&junk, sizeof(junk), 0);
+ ASSERT_EQ(save(ANT_SNAPSHOT_POOL_FILENAME, sizeof(junk), (unsigned char*)&junk),
+ (long long)sizeof(junk));
+
+ EXPECT_FALSE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK));
+
+ // The pool is read after the meta checks pass, so a refusal here does clear the colony.
+ EXPECT_EQ(colony->solutionCount(), 0u);
+}
+
+// ---------------------------------------------------------------------------------------------
+// Replay cache
+
+// A key whose four components are all distinct, so a slot function that ignores one still separates
+// these.
+static AntColonyBpp9000T::ReplayKey makeReplayKey(unsigned long long n)
+{
+ AntColonyBpp9000T::ReplayKey k;
+ k.pubkey = makeKey(n);
+ k.nonce = makeKey(n + 1000);
+ k.parentAnnHash = makeKey(n + 2000);
+ k.anchorDigest = makeKey(n + 3000);
+ return k;
+}
+
+// lut[0] carries n so two networks are distinguishable; the rest stays a legal trit.
+static AntColonyBpp9000T::Ann makeAnn(unsigned char n)
+{
+ AntColonyBpp9000T::Ann a;
+ setMem(&a, sizeof(a), 0);
+ a.lut[0] = (unsigned char)(n % 3);
+ a.lut[1] = (unsigned char)((n / 3) % 3);
+ return a;
+}
+
+static bool annEquals(const AntColonyBpp9000T::Ann& a, const AntColonyBpp9000T::Ann& b)
+{
+ for (unsigned long long i = 0; i < sizeof(a); i++)
+ {
+ if (a.lut[i] != b.lut[i])
+ {
+ return false;
+ }
+ }
+ return true;
+}
+
+// The score and the network both come back. The network matters as much as the score: commit()
+// stores it and childAnnHash folds it into resourceTestingDigest.
+TEST(TestAntColonyReplayCache, StoresAndReturnsScoreAndNetwork)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ const AntColonyBpp9000T::ReplayKey key = makeReplayKey(1);
+ const AntColonyBpp9000T::Ann ann = makeAnn(7);
+ colony->putReplayScore(key, 3800, ann);
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ setMem(&out, sizeof(out), 0xFF);
+ ASSERT_TRUE(colony->tryGetReplayScore(key, score, out));
+ EXPECT_EQ(score, 3800u);
+ EXPECT_TRUE(annEquals(out, ann));
+}
+
+// Every component is part of the key, so changing any one of them must miss. Missing one would
+// return a score computed from different inputs.
+TEST(TestAntColonyReplayCache, EveryKeyComponentIsPartOfTheLookup)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ const AntColonyBpp9000T::ReplayKey key = makeReplayKey(2);
+ colony->putReplayScore(key, 3800, makeAnn(1));
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ for (int component = 0; component < 4; component++)
+ {
+ AntColonyBpp9000T::ReplayKey altered = key;
+ switch (component)
+ {
+ case 0: altered.pubkey = makeKey(90001); break;
+ case 1: altered.nonce = makeKey(90002); break;
+ case 2: altered.parentAnnHash = makeKey(90003); break;
+ case 3: altered.anchorDigest = makeKey(90004); break;
+ }
+ EXPECT_FALSE(colony->tryGetReplayScore(altered, score, out)) << "component " << component;
+ }
+ EXPECT_TRUE(colony->tryGetReplayScore(key, score, out));
+}
+
+// A new epoch changes every root and anchor digest, so no entry could hit anyway; keeping them would
+// just hold slots.
+TEST(TestAntColonyReplayCache, BeginEpochClearsIt)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ const AntColonyBpp9000T::ReplayKey key = makeReplayKey(3);
+ colony->putReplayScore(key, 3800, makeAnn(2));
+ ASSERT_EQ(colony->replayCacheOccupancy(), 1u);
+
+ colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK);
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ EXPECT_FALSE(colony->tryGetReplayScore(key, score, out));
+ EXPECT_EQ(colony->replayCacheOccupancy(), 0u);
+}
+
+// loadSnapshot() calls reset(), and the catch-up that follows a restore is the one moment the cache
+// is worth most. Losing it there would defeat the feature.
+TEST(TestAntColonyReplayCache, SurvivesResetAndSnapshotLoad)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ const AntColonyBpp9000T::ReplayKey key = makeReplayKey(4);
+ colony->putReplayScore(key, 3800, makeAnn(3));
+ ASSERT_TRUE(colony->saveSnapshot(TEST_EPOCH, NULL, TEST_INITIAL_TICK));
+ ASSERT_TRUE(colony->loadSnapshot(TEST_EPOCH, NULL, TEST_ROOT_SEED, TEST_THRESHOLD, TEST_INITIAL_TICK));
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ EXPECT_TRUE(colony->tryGetReplayScore(key, score, out));
+ EXPECT_EQ(score, 3800u);
+}
+
+// The file is the table verbatim, so this checks that entries survive the write and stay findable
+// under their own keys. Enough of them that collisions and evictions are in play. One save writes
+// the whole ANT_REPLAY_CACHE_BYTES table, so this is the only test here that touches a file.
+TEST(TestAntColonyReplayCache, RoundTripsThroughAFile)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ constexpr unsigned int COUNT = 500;
+ for (unsigned int i = 0; i < COUNT; i++)
+ {
+ colony->putReplayScore(makeReplayKey(10000 + i), 3000 + i, makeAnn((unsigned char)i));
+ }
+ ASSERT_EQ(colony->replayCacheOccupancy(), COUNT);
+ ASSERT_TRUE(colony->saveReplayCache(TEST_EPOCH, NULL));
+
+ colony->clearReplayCache();
+ ASSERT_EQ(colony->replayCacheOccupancy(), 0u);
+
+ ASSERT_TRUE(colony->loadReplayCache(TEST_EPOCH, NULL));
+ EXPECT_EQ(colony->replayCacheOccupancy(), COUNT);
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ for (unsigned int i = 0; i < COUNT; i++)
+ {
+ ASSERT_TRUE(colony->tryGetReplayScore(makeReplayKey(10000 + i), score, out)) << "entry " << i;
+ ASSERT_EQ(score, 3000 + i) << "entry " << i;
+ ASSERT_TRUE(annEquals(out, makeAnn((unsigned char)i))) << "entry " << i;
+ }
+}
+
+// No cache is the normal state at the start of an epoch, so it must report a miss and leave an empty
+// table rather than fail the boot.
+TEST(TestAntColonyReplayCache, AbsentFileIsNotAnError)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.9 GB";
+
+ colony->putReplayScore(makeReplayKey(7), 3800, makeAnn(6));
+ EXPECT_FALSE(colony->loadReplayCache((unsigned short)(TEST_EPOCH + 77), NULL));
+ EXPECT_EQ(colony->replayCacheOccupancy(), 0u);
+
+ unsigned int score = 0;
+ AntColonyBpp9000T::Ann out;
+ EXPECT_FALSE(colony->tryGetReplayScore(makeReplayKey(7), score, out));
+}
+
+// ---------------------------------------------------------------------------------------------
+// Export best ANN at the end of epoch
+
+// The file layout: one header, then entryCount of these.
+struct ExportFileEntry
+{
+ AntColonyExportEntry meta;
+ AntColonyBpp9000T::Ann ann;
+};
+
+// Reads antColonySolutions.eoe back. Header first, since only it says how long the body is.
+static bool readExport(AntColonyExportHeader& header, std::vector& entries)
+{
+ if (load(ANT_COLONY_SOLUTIONS_EOE_FILENAME, sizeof(header), (unsigned char*)&header)
+ != (long long)sizeof(header))
+ {
+ return false;
+ }
+ entries.clear();
+ if (header.entryCount == 0)
+ {
+ return true;
+ }
+
+ const unsigned long long total = sizeof(header)
+ + (unsigned long long)header.entryCount * sizeof(ExportFileEntry);
+ std::vector raw(total);
+ if (load(ANT_COLONY_SOLUTIONS_EOE_FILENAME, total, raw.data()) != (long long)total)
+ {
+ return false;
+ }
+ entries.resize(header.entryCount);
+ copyMem(entries.data(), raw.data() + sizeof(header), total - sizeof(header));
+ return true;
+}
+
+// More solutions than the file holds, so the cap, the eviction and the ordering are all exercised.
+TEST(TestAntColonyExport, KeepsTheLowestScoresInOrder)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ constexpr unsigned int COMMITTED = ANT_EXPORT_MAX_SOLUTIONS + 24;
+ // One identity per solution so the per-parent child cap never binds - the export set is what is
+ // under test here, not the tree shape.
+ for (unsigned int i = 0; i < COMMITTED; i++)
+ {
+ ASSERT_NE(commitRootChild(colony, makeKey(1 + i), 3000 + i, i, 5000 + i), ANT_INVALID_INDEX) << "commit " << i;
+ }
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+
+ EXPECT_EQ(header.entryCount, ANT_EXPORT_MAX_SOLUTIONS);
+ EXPECT_EQ(header.solutionCount, COMMITTED);
+ EXPECT_EQ(header.entrySizeBytes, (unsigned int)sizeof(AntColonyExportEntry));
+ EXPECT_EQ(header.annSizeBytes, (unsigned int)sizeof(AntColonyBpp9000T::Ann));
+ ASSERT_EQ(entries.size(), (size_t)ANT_EXPORT_MAX_SOLUTIONS);
+
+ // The 676 lowest of 3000..3699, so exactly 3000..3675, ascending.
+ EXPECT_EQ(entries[0].meta.score, 3000u) << "entry 0 must be the best network of the epoch";
+ EXPECT_EQ(entries[ANT_EXPORT_MAX_SOLUTIONS - 1].meta.score, 3000u + ANT_EXPORT_MAX_SOLUTIONS - 1);
+ for (unsigned int i = 1; i < entries.size(); i++)
+ {
+ ASSERT_LE(entries[i - 1].meta.score, entries[i].meta.score) << "not ascending at " << i;
+ }
+}
+
+// Below the cap the file holds everything, still ordered - and the scores are committed descending
+// here, so every insert lands at the front and the shift path is the one being used.
+TEST(TestAntColonyExport, OrdersFewerThanTheCap)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ constexpr unsigned int COUNT = 40;
+ // One identity per solution so the per-parent child cap never binds.
+ for (unsigned int i = 0; i < COUNT; i++)
+ {
+ ASSERT_NE(commitRootChild(colony, makeKey(2000 + i), 3800 - i, i, 6000 + i), ANT_INVALID_INDEX);
+ }
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+
+ ASSERT_EQ(header.entryCount, COUNT);
+ EXPECT_EQ(entries[0].meta.score, 3800u - (COUNT - 1));
+ for (unsigned int i = 1; i < entries.size(); i++)
+ {
+ ASSERT_LE(entries[i - 1].meta.score, entries[i].meta.score) << "not ascending at " << i;
+ }
+}
+
+// Equal scores keep the incumbent, so the earlier solution ranks first. Without a total order two
+// nodes with the same solutions could write different files.
+TEST(TestAntColonyExport, TiesKeepTheEarlierSolution)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ const m256i first = makeKey(3);
+ const m256i second = makeKey(4);
+ ASSERT_NE(commitRootChild(colony, first, 3500, 0, 6100), ANT_INVALID_INDEX);
+ ASSERT_NE(commitRootChild(colony, second, 3500, 1, 6101), ANT_INVALID_INDEX);
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+
+ ASSERT_EQ(header.entryCount, 2u);
+ EXPECT_TRUE(entries[0].meta.pubkey == first);
+ EXPECT_TRUE(entries[1].meta.pubkey == second);
+}
+
+// The stored network has to survive the round trip, a wrong ANN here is a wrong harvest, and
+// nothing downstream would notice.
+TEST(TestAntColonyExport, CarriesTheNetworkAndItsDepth)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ const m256i me = makeKey(5);
+ ASSERT_NE(commitRootChild(colony, me, 3800, 0, 6200), ANT_INVALID_INDEX);
+ const SolutionRef aRef = { TEST_PUBLISH_TICK,0 };
+ ASSERT_NE(commitChild(colony, me, aRef, 3700, 1, 6201), ANT_INVALID_INDEX);
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+ ASSERT_EQ(header.entryCount, 2u);
+
+ // Best first: the depth-2 child at 3700, then its depth-1 parent at 3800.
+ EXPECT_EQ(entries[0].meta.score, 3700u);
+ EXPECT_EQ(entries[0].meta.depth, 2u);
+ EXPECT_EQ(entries[1].meta.score, 3800u);
+ EXPECT_EQ(entries[1].meta.depth, 1u);
+
+ AntColonyBpp9000T::Ann expected;
+ ASSERT_TRUE(colony->annOfNonRoot(*colony->recordAt(1), expected));
+ for (unsigned long long i = 0; i < sizeof(expected); i++)
+ {
+ ASSERT_EQ(entries[0].ann.lut[i], expected.lut[i]) << "genome byte " << i;
+ }
+}
+
+// The set holds networks the records cannot reproduce once the store is full, so it is saved rather
+// than rebuilt. If that file went missing the export would come back empty after a restart.
+TEST(TestAntColonyExport, SurvivesASnapshot)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ const m256i me = makeKey(6);
+ for (unsigned int i = 0; i < 10; i++)
+ {
+ ASSERT_NE(commitRootChild(colony, me, 3700 + i, i, 6300 + i), ANT_INVALID_INDEX);
+ }
+ ASSERT_TRUE(saveWipeLoad(colony));
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+
+ ASSERT_EQ(header.entryCount, 10u);
+ EXPECT_EQ(entries[0].meta.score, 3700u);
+ EXPECT_EQ(entries[9].meta.score, 3709u);
+}
+
+// A new epoch starts with nothing to export, and the file must say so rather than carry last epoch's.
+TEST(TestAntColonyExport, BeginEpochClearsIt)
+{
+ AntColonyBpp9000T* colony = freshColony();
+ ASSERT_NE(colony, nullptr) << "colony init failed; needs ~6.6 GB";
+
+ ASSERT_NE(commitRootChild(colony, makeKey(7), 3500, 0, 6400), ANT_INVALID_INDEX);
+ colony->beginEpoch(TEST_ROOT_SEED, TEST_INITIAL_TICK);
+ ASSERT_TRUE(colony->exportBestSolutions(TEST_EPOCH, NULL));
+
+ AntColonyExportHeader header;
+ std::vector entries;
+ ASSERT_TRUE(readExport(header, entries));
+ EXPECT_EQ(header.entryCount, 0u);
+ EXPECT_EQ(header.solutionCount, 0u);
+}
diff --git a/test/ant_pending_solutions.cpp b/test/ant_pending_solutions.cpp
new file mode 100644
index 000000000..d26c084cf
--- /dev/null
+++ b/test/ant_pending_solutions.cpp
@@ -0,0 +1,233 @@
+#define NO_UEFI
+
+#include "gtest/gtest.h"
+
+#include "../src/mining/ant_colony/ant_pending_solutions.h"
+
+static m256i key(unsigned long long n)
+{
+ m256i k = m256i::zero();
+ k.m256i_u64[0] = n + 1; // never zero: a zero pubkey marks an unused slot
+ return k;
+}
+
+static SolutionRef ref(unsigned int tick, unsigned int idx)
+{
+ SolutionRef r;
+ r.tick = tick;
+ r.solutionIndexInTick = idx;
+ return r;
+}
+
+// 5.75 MB, so one buffer for the file, reset between tests.
+static AntPendingSolutions* freshPool()
+{
+ static AntPendingSolutions pool;
+ static bool allocated = false;
+ static bool failed = false;
+ if (!allocated && !failed)
+ {
+ failed = !pool.init();
+ allocated = !failed;
+ }
+ if (failed)
+ {
+ return nullptr;
+ }
+ pool.reset();
+ return &pool;
+}
+
+// The key is (computor, parentRef, nonce). Same triple twice is one solution, whatever else differs.
+TEST(TestAntColonyPending, DedupsOnTheConsensusKey)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+
+ EXPECT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ // A different anchor is the SAME solution - anchorTick is deliberately not in the key, so a
+ // re-anchored resend cannot be published twice.
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 9999, 0, key(900)));
+
+ // Any other field differing makes it a different solution.
+ EXPECT_TRUE(pool->add(key(2), ref(100, 0), 5000, 0, key(900)));
+ EXPECT_TRUE(pool->add(key(1), ref(100, 1), 5000, 0, key(900)));
+ EXPECT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(901)));
+}
+
+// A fresh entry is selectable, and only by the computor it belongs to.
+TEST(TestAntColonyPending, SelectsOnlyForItsOwnComputor)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ AntPendingSolution out;
+ EXPECT_EQ(pool->selectForPublish(key(2), 5000, out), AntPendingSolutions::NO_ENTRY);
+
+ const unsigned int idx = pool->selectForPublish(key(1), 5000, out);
+ ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY);
+ EXPECT_TRUE(out.nonce == key(900));
+ EXPECT_EQ(out.anchorTick, 5000u);
+}
+
+// Scheduling records a deadline. Before it passes the entry must not come back, or the node would
+// republish a transaction that is still in flight.
+TEST(TestAntColonyPending, ScheduledEntryIsNotReselectedBeforeItsDeadline)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ AntPendingSolution out;
+ const unsigned int idx = pool->selectForPublish(key(1), 5000, out);
+ ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY);
+ pool->markScheduled(idx, 5003);
+
+ EXPECT_EQ(pool->selectForPublish(key(1), 5001, out), AntPendingSolutions::NO_ENTRY);
+ EXPECT_EQ(pool->selectForPublish(key(1), 5002, out), AntPendingSolutions::NO_ENTRY);
+
+ // Deadline reached with no acknowledgement: republish.
+ EXPECT_EQ(pool->selectForPublish(key(1), 5003, out), idx);
+}
+
+// The whole reason the state is a tick and not a flag.
+TEST(TestAntColonyPending, RetriesOutrankFreshEntries)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+ AntPendingSolution out;
+ const unsigned int stale = pool->selectForPublish(key(1), 5000, out);
+ ASSERT_NE(stale, AntPendingSolutions::NO_ENTRY);
+ pool->markScheduled(stale, 5003);
+
+ // Newer solutions keep arriving while the first one's transaction is lost.
+ ASSERT_TRUE(pool->add(key(1), ref(100, 1), 5001, 0, key(901)));
+ ASSERT_TRUE(pool->add(key(1), ref(100, 2), 5002, 0, key(902)));
+
+ // Past the deadline the retry must win, or a steady stream of new work starves it forever.
+ EXPECT_EQ(pool->selectForPublish(key(1), 5003, out), stale);
+}
+
+// RECORDED comes from observing the chain, and must both stop republication and suppress a resend.
+TEST(TestAntColonyPending, RecordedStopsRepublishingAndSuppressesResend)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ AntPendingSolution out;
+ const unsigned int idx = pool->selectForPublish(key(1), 5000, out);
+ ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY);
+ pool->markScheduled(idx, 5003);
+ pool->markRecorded(key(1), ref(100, 0), key(900));
+
+ EXPECT_EQ(pool->selectForPublish(key(1), 9000, out), AntPendingSolutions::NO_ENTRY);
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+}
+
+// A transaction the node never queued still has to be remembered, or a miner resending a solution
+// that is already on-chain makes the pool publish it a second time and pay a second deposit.
+TEST(TestAntColonyPending, RecordingAnUnqueuedSolutionSuppressesALaterSubmission)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+
+ pool->markRecorded(key(1), ref(100, 0), key(900));
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+}
+
+// Past the publish window the commit path rejects it as stale, so publishing spends the deposit for
+// nothing. Selection has to drop it rather than hand it over.
+TEST(TestAntColonyPending, ExpiredEntriesAreRetiredNotPublished)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ AntPendingSolution out;
+ EXPECT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS, out), 0u);
+ EXPECT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS + 1, out), AntPendingSolutions::NO_ENTRY);
+
+ AntPendingSolutions::Stats stats;
+ unsigned int count = 0;
+ pool->getStats(stats, count);
+ EXPECT_EQ(stats.obsoleteExpired, 1u);
+}
+
+// An entry retired for expiry was never published, so the seen filter was never marked and the key
+// is still usable. A resubmission with a fresh anchor must replace it rather than be refused as a
+// duplicate - otherwise the solution is stranded and the miner is never told why.
+TEST(TestAntColonyPending, ExpiredEntryCanBeResubmittedWithAFreshAnchor)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ // Selection retires it instead of publishing: publishing a stale one would mark the seen filter
+ // and kill this key permanently.
+ AntPendingSolution out;
+ ASSERT_EQ(pool->selectForPublish(key(1), 5000 + ANT_PUBLISH_WINDOW_TICKS + 1, out), AntPendingSolutions::NO_ENTRY);
+
+ // Same triple, newer anchor. This is the replacement, not a duplicate.
+ EXPECT_TRUE(pool->add(key(1), ref(100, 0), 40000, 0, key(900)));
+
+ const unsigned int idx = pool->selectForPublish(key(1), 40000, out);
+ ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY);
+ EXPECT_EQ(out.anchorTick, 40000u);
+}
+
+// The score is computed once, at receipt, and the publisher reads it back from the entry.
+TEST(TestAntColonyPending, CarriesTheScore)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 3771, key(900)));
+
+ AntPendingSolution out;
+ ASSERT_NE(pool->selectForPublish(key(1), 5000, out), AntPendingSolutions::NO_ENTRY);
+ EXPECT_EQ(out.score, 3771u);
+}
+
+// A live entry is a real duplicate, whether it has been scheduled or not.
+TEST(TestAntColonyPending, LiveEntryStillRejectsAResubmission)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+ ASSERT_TRUE(pool->add(key(1), ref(100, 0), 5000, 0, key(900)));
+
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 6000, 0, key(900)));
+
+ AntPendingSolution out;
+ const unsigned int idx = pool->selectForPublish(key(1), 5000, out);
+ ASSERT_NE(idx, AntPendingSolutions::NO_ENTRY);
+ pool->markScheduled(idx, 5003);
+ EXPECT_FALSE(pool->add(key(1), ref(100, 0), 6000, 0, key(900)));
+}
+
+// Finished slots are reused in place, so a pool that has published for a whole epoch does not fill.
+TEST(TestAntColonyPending, FinishedSlotsAreReclaimed)
+{
+ AntPendingSolutions* pool = freshPool();
+ ASSERT_NE(pool, nullptr);
+
+ for (unsigned int i = 0; i < 4; i++)
+ {
+ ASSERT_TRUE(pool->add(key(1), ref(100, i), 5000, 0, key(900 + i)));
+ pool->markRecorded(key(1), ref(100, i), key(900 + i));
+ }
+
+ AntPendingSolutions::Stats stats;
+ unsigned int count = 0;
+ pool->getStats(stats, count);
+ EXPECT_EQ(stats.recorded, 4u);
+
+ // Nothing left to publish, and new work still fits.
+ AntPendingSolution out;
+ EXPECT_EQ(pool->selectForPublish(key(1), 5000, out), AntPendingSolutions::NO_ENTRY);
+ EXPECT_TRUE(pool->add(key(1), ref(200, 0), 5000, 0, key(1000)));
+}
diff --git a/test/data/bpp9000.task b/test/data/bpp9000.task
new file mode 100644
index 000000000..8ffa5e17a
Binary files /dev/null and b/test/data/bpp9000.task differ
diff --git a/test/data/gt_ant_production.csv b/test/data/gt_ant_production.csv
new file mode 100644
index 000000000..664c75749
--- /dev/null
+++ b/test/data/gt_ant_production.csv
@@ -0,0 +1,65 @@
+chain, depth, pubkey, nonce, anchor, seed, score
+2, 0, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 01084ed07b9200fe2bb401df72b52ba819fd5c09e436f5f185594f8b8afcbc66, 54ba8fded70f55d660977d169d6a2ab6d7711b11f2b49596fbab1d5bf968a961, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4576
+5, 0, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 0106216c6da947657f74a629a59d5b81da7a4a6ecce8a3abb4b23079b5f1c56e, 21c6ec71261f8027ee7fe4629982e12c50facf8d0acc1f8f297264db6bc2dc2c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 6400
+11, 0, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01085ebfe74c8324c42214dde9c3c71f691e327cd8a764e4c396d7431deaae03, 12545454084db67b036475332b9a7ebd7c7ed73b13e64bc17d87ec2e758dcb22, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5308
+1, 0, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 010523bbe90fa21b263946830cc53e8744e270cf6e944b1cf255e85ac74a7887, eae546338da0b54a3b4e10f859b6293f1d285a0e51e4881f10d1379ff511f819, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 6295
+0, 0, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 01023378b9bf89fae25fdcd165c8739da9dbba04ff92982345a623031d23b0fb, 91a1d4fc06689515127b01f116fd64c4c2ca4c02a4692675c1b0dd4fc1bf3589, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4800
+8, 0, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010733ccce2e806a4aea54c0dcc386ebfd4efc6f3027f63c1de301686508f403, a2050b44a20027fb98984b156632263a8bccefd34e4c1a06d516ac807ea19433, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4414
+6, 0, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 010a33855fd20d0f1c910f0f7f7bc4eef7dfde506c7184d12bb6e50dc095b9de, 0509c75356fc65087c969b8eb8602a28c903a34d8f0648394e29dbeaa9cb4892, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206
+10, 0, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01080bf6551a06f4826c9dfb047814381042f281b322ab9452623183daab9dd3, a0651819278fad70d8d2e24db8fa595f305cb879b400fb0c270ecad1361cc25c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4352
+4, 0, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 0105337415ffe5aae3f1e4816712ba7ca6e4df3db6b10276868d4e2861cb69e8, a18598ffcfd37339a922658ea450e94f2e4cd083c4e64d5013d002c4fcb17743, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4379
+6, 1, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 010a54a6afb54d321417a986b3e8ce05e51823ea0aceef2681392fcda42010b0, bb849545500e6716bdbbf6fbbdcf6bfe1fd2b4cc20c1024cb419be77dae27df0, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206
+5, 1, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 010354eaf1f371ea113434301d5e15b04d24b92fc11dd20f5e03fb6e6f497d30, e9bf8e92a20da221725f0897cf111f326c7754ead360f7aba074459ad99ec7f9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5998
+3, 0, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 010329371769a20be8033c7868d60327ad8b345b190f69f6a6b54f55c7acde8c, a3e0ad1361d551b2ce13025624602a4ebbcb0bc37c93348a88d23d394c155bd9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4569
+2, 1, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 01081abdbc10a2b70d301fe0bb0afa298f90a5d9bdd0777d24949228abc63106, 02724836ecf8a9bd54e8f78867ea5b4e5c13dc6c41566094a6b41fa36e219f09, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4576
+8, 1, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010229cda723c097587f19e30ee090700b7da9bfc9a24e2fb73d83d55b4ba107, c55bea5399c1ca80efe9b9b6c51991ce99b6cc34f804984957cc7a110a1e03cd, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4373
+0, 1, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 010615c66cb4c65a74382fce9000de43248a4425ced3bc79929139502e97c705, 4a564c2984b914fda7e68a1c80fd1bfdf6a75ac2443f348e98645d42fbaa052a, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450
+10, 1, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 010a5e540d6e35170556e51d0a25400b8e7dce3a21872d56e397633411fc7656, eb23ade51e8662688b5e069f59e64c9049c7896058ee4460bab704180f8be6e6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4352
+1, 1, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 01050a3c2127a028962e38ab80d9eb6848790106a3809637f8b1eb849c261a1b, 8dc4c241e049f0be5341d03b0d1023ae5340cd8cc506dd6f03a998743e2fed75, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663
+2, 2, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 010246c5bb2fa79a2bc5f2ed1fbb2877ba2de90011ee439383347c684cfb80a6, ef9bab5a83f63baadbd79d9e3680a36eeb65d8c9c1c52c6dce0e47f807ccb465, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4394
+11, 1, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01025ded1ea31c0763dd89595fce37c703c886203ef104ccea9979bf6041b511, 967e0547550da817764ba0a7b51e33556c2435e9a910e4b16d9300bd785a06e1, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5308
+4, 1, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 0106282d2480193df99350d4ef0c3b31b9a4314b38cc3d0e86c3c9564b9fc77b, 9ba91514fec4c80a40201e7fbc9bf7d08ee446b8f3efaff16e32ae54c5afdc92, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4310
+6, 2, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 01052e993e876de4beeed6b7e5f674acf08d15f44a41dc59e080b50b5e9ee9ee, a570a04a90af0b23bf9ea5b4be54314a853f517a060c39df1d983e9665f4c45b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206
+3, 1, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 0109595f50ab0a4d9c0c2209dc2e020058a6a3fcaa4ab687f5338d46a4259031, b8a420a950c50a97f8876a589cc1a4f2822eed80efc2e7ef6bd44b2c780061c6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4569
+5, 2, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 01030cc6084cbef2f92c8fc481db9b3b36ec91723770bb5c6aaea1dfb2104376, b3310ce38053f82c73d26eed58ad00897a8d0de995d60c4eeb8d4755afc518c9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4836
+10, 2, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01040809aeb747d430f914d0ceb1970d3ff7e50b44ac1db1de8ac6a910795423, 5df3e71634f680908f5954ce538b65f470d65779dfcae051f1d42b43f1527abe, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4241
+10, 3, eda9ead70a463d6bf5858faf2a8caa3c3d20b3c3e0e994167e685200bc750276, 01075358c48d3c17682f8c69578d2db1710168ebad922b591b764b0937353be2, 190832364108015065a051e38badc3535be4faff42a319132ebfecea4bf03416, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4241
+8, 2, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 010307ae588e0d66befcfb11099713189200bfeab97cb37d4f18bfa0d877cd2a, 2032dcba5a6c53d1cb68ad800ae8c7b455aef9aa90e1c14cb5591bbe2c8e1927, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4307
+6, 3, dc40c51265eff259556066d427dbeebb4109a7e79690d61d6349ef650b29dcb9, 01014f1dc6bcc0311403fe50177fe2b69d702b1109db30aab04be2b6cd964375, 7f2fbd2dca697fee9169049bd06a5cadb6932db37cc8190b7f781e2859a13ddf, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4206
+0, 2, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 01011280b730b2317044f434b8e56ca881f7efeaf8dd4b29b12e877ea43fb7a1, d260aab327663b1197f602584a74c80dfdc3915d487440e21d59b6d795ba284c, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450
+2, 3, 886ef7f14d42287f8a74aaa92a409eaf901e25b6f9cfbedb5f72e1297bcad551, 0107159b5267736895605a21efe7bfa20f676707dd2bfdccabc6d67ed211d511, 32fa4cc87d7f7ba35dfcc9fad0256dd64bd1c3a340e1fb1ce92a2e3102641be8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4291
+11, 2, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 01040f59f2e3d85b897c1997ae4cd6a1aa20f0397795106ec35742efa2f60207, 70efb4b8a77579ec350179a71f89e517c378ec87896a526d5bca985e45e6a2fc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4581
+1, 2, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 0103333ea8e0bc20c0f2cfb7cc93e9d3ae302f4b5e933e605ca8d5f2b101f7c8, 39cb97cbe391fff147ecffe6bc7b6cfcf920726974c9ad5394b54e7a86d3e752, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663
+14, 0, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 0103553c86f3dcc0e0a8c63a9004bc5719b09b04348d77a9de594f5b8bdffaae, db7e8851e5e2b45c072bf7cd15040a9bc313b6d4e8e568f7a50a62810a181842, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5808
+5, 3, 2df29076f8969739636dff083eb1f54bfe7e5d8fcac5b2946d0871d40ba495aa, 01022aa2e9c6bf0d46e240165a772be18804f052b8f76111e0c22af538b332e6, ed0147d60ab884ab994eca1b25f319349437e4106b21068d0f8ae9c357102e14, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4779
+4, 2, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 01080ccfc0cce11c4229f4dd023630330c17f3f495c3db73a30ed50107882a5e, eb7729568d616b19004a42dab7f8b237defd6eb3a0a5448037ecf4ec56044bd7, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4137
+3, 2, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 01013064dacd09cf7f1898ac0b669d25f4c0db3c2bcea9b3c9cd51f3fc8a7414, 5a107488a0caf086ae94acfefca9b36f6ae84691b11a001ef3999a673d7b160b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4394
+0, 3, 4657de22da43ad6402c8cfdd45ba54542213486481988bdd28c82182ec2714af, 0109151f9d11847474e333460ea73646a31663c84162313db3cea328a74efa82, c10184404849ef460dd734d49d22cad1b25aafa77bbd3f22073dc88885f644ec, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4450
+4, 3, e5e05648b1c70c4deea4c7d7aefccd49b24defadceb45dd033caf0f16407808b, 01044b182869be4ba5670092661abac68606a2f4b06130d101882d236e482358, b7fc5ca1cb869e33b3725a1f3a4c96e8545200d0f263d099351016b6baf46039, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4137
+14, 1, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 01076440117d3ff4d99c831ad310105cb1217c9bfe4bbd7a8d98df63fde9dac1, c01ee3f86edc2dbb38b5fcabf8dca2e5f63e80cfc7161b897783b71fb24eb07b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5808
+8, 3, e0b3b0f9b66fe475eaf17c6be38a2759ff254debd5485c4d4a27bc45486fa0ee, 01092583572dbd57774dc64402af3254c4e9cfc97a76b686d690e728f4946882, c1003cacbb633a682fa1587cc0a59535501e3d8511b896df7431c320da790724, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4294
+11, 3, 83b379182516db53587b3c553468e1af0ca4c21f13b95150024e68ebb2bb9550, 010522c21f13bb19c26a46e5798d2ce597bfd035261c4a3e9e2d38963910dba2, e692e800bb6e620f5108e70950e6f65c46997a3dd8aef402a30d44870ba94440, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4344
+12, 0, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 010618c41d7d80fdeb077f54165537de721e39ee32e5f8d11dd16e5edff70d91, 9c65303cf21df1f289cd5d7e1d431ddd79016d87c6e8ec57b2c23c15c69c119f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862
+1, 3, 17620b9920ef596dfa46753b355247b1641eaeda1246fa7bc43884630cf61bf2, 0108598ac27cff46dd5635d43a3ba7b32bf7c0f844c95418fb359f4bb8b14a48, cdb0470fe9f772124a6d158869b4bed2451b4445293ffa2cc56f96917b108ebc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4663
+12, 1, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 010a33039aba1207b1d63d559fbe40220eaa30b56fd25e5d0767eed2538ed35c, 27ee5997236575efdb19c9845b330da3e969ab8b69af32b6cbace636afbca8cc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862
+13, 0, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 010a602d4d374fe77a551445a50be97aa8ff00a8e6d1bc87eb6a05a7ca9d5bac, 5e65aac82ee69c59e00bd8bc6b688b45c6a30fc14a97c50f631f6255eaaf7327, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4916
+14, 2, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 010203df7f6c6ec32fb907ddd1bfe342d0e638248cf885e02534d40ee5b0fdc7, 7f25490bb25c2a15a4ae4b09834af66b32f6c747cd410f205ebb5f26157628b3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5475
+3, 3, 072fef44ac2f7667c366f067b374016bdd39581dbe8acb0f8aadf98502d13646, 01011fba8c525ade36b44b91c10f763cc452b9f359138a17b7095defba3de81f, ddbcfaeaeefae22924db28ad1ac6dbab0878009449fa125d1e072edfd29bfe29, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4337
+12, 2, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 01081ae266672ca194549f553909f744cc0043134958ecc17e1cb23b8b321b81, eabf4471ea17eeb0e6b8ef2868bdf533e538e9117d89ee7be0506b5b87100c31, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4862
+13, 1, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 01061e74c28eab539b083cc334b3d8f49d500d3a0efaf0ddacfeac3c3d7db59c, a1f90ec26c1e922437b5dce2cb7d5aa74a73dd6b2a633e0aa1a87e181ef4e493, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183
+12, 3, 2814b082c6683e9490338df3e42c8d89ca9e2f889735013dcbb7ddbdf7d846c4, 0107093cee10017f16cccbff4c1bf264ad41d8416654ce13f71d06d40a068d45, 4728ca3016635dbb5fed672ad264689e97f39372d87fdc8751d24d305147e50f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4808
+15, 0, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 010a4622db8bdc6f2d2a53c4d093a7db2eed1511435c0c9d5f3ad99ccc89461f, 782f1d06c5a143065b84a674c8c957ec1f6c335b9b9d1818aa440358490f9d42, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5165
+15, 1, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 010758cd460961f2865307025c35b5c371a152aa887aa79a5ed5db1562554b51, d949da8e7b84d066ea4e0d44064dfc4d8c58b4c4f6580a817de564a74500ce2e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5165
+14, 3, 8c2cce72fa295e618dada27cd9dce2a8f24a4596d676edf11e096bcf6f5c078f, 01051655e2f51a514371801ef25660a391477f0d6c492b613c5a83bb30f490f6, 66473bfc883091c7d6b1ddcb8753db715e7a9568aca4b2f870f155edf0b952b3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4839
+15, 2, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 01034a2930d5d739d27f0a4b65bee1bd19ee6dccc2da5612aa3e380d60e263a3, 61cda7fa7dd9cb6b817139a51b7d17f3fef1ddb31db2a506082cd49411d0df57, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4541
+13, 2, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 0102482e077b5bbcf0c1f5da9a77803663ade42ccf6667a2d07366ec25c05373, 7cdb704cdc5df918699d7bd399008f249f79ab8b4ca08689f8d3010c5813de38, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183
+15, 3, dac78edf70e526e26a94ac4982c2abded02fe049c5038196738d66d80849ca3d, 0104501feb2927fd13760f5c39dc277827c125b069804c6ac18e6021b1811868, 61078520e97db6e828ce6465404fe3456dc27deb16516cfdc3c34c8293c52fd0, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4541
+13, 3, e36507d27632e2d6cc70480f6322a4469d20773ff0d047f6f70285e9d44beb5b, 01024f0896557a0084a1d9a2937130ccb2711a8cb810e512ec9e2c97c3c56db6, d4cbdaa2fb4e3c1e8ef809ed628f0abf16d3bf4a93f81ce03bf77b68e07fb516, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4183
+9, 0, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 0102152a605e81991fe51e4c8964a98462f6a78ab03dfabfd30552d39e9d4eeb, c17c5c2242fd08302a00dddf39bfe32952a319f4816dbc540b991d8ee33180b4, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5099
+9, 1, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 01012c56d755fe8e1c3c18ce6c13e4fb74a62f4dbf6b4d88a4d4d5f12a82111f, aec69cdd848a1b7ff0f89c9ea97feb759270ed6719ec8a675ceba9cc0877a1de, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5035
+9, 2, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 010918e449d18c7ea382562025280bd33f3dea08e299eb92eaa7413d9b42c367, 1cb9b9ffb834abc7b8bbfeb01e006fd3cc412717f8b9456e5e4d663466417e99, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4281
+9, 3, a14d543d43592ce38ef38f600a2dba801557fd924638de966d2725dabc5e2472, 010a107f47f98b2357d358f56f2a9cb9759714f3bec8ae28dee41e9ce170a5db, c6d51cd991be84979a28aa4b4e51de3ef0f585aadbab7b66a0593805a5085546, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4132
+7, 0, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 010303ab82e286a8adbe84e8b7d72f1c627058d749eb7fee957227ae354736a9, e1b75477f812f034862401a8ecf7c9b7ffed63e904ebff99b4662bb3157d55f7, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4382
+7, 1, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 01061f0f60aa46bb3d3db810b004f0d63a0666f4304c7902c2d0de7116fd4c3a, a6453d96ed2963d2f8073faf29405223c3b9de7b9f0bf925ad5b5807f0df29ec, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4382
+7, 2, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 010a01e51b9910f599cf59ad2c563f0dedb61dbc3c0226bad673b48d1b538419, 1485b5332779be0d64bd08c69344580358b320cf01b9f38ce70badc069e94ef1, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032
+7, 3, 158b59e2f807c4b05160ad5dae50f96f079b79a7f7467e11d86c37a808515c1a, 01050c4baf9613939235ebdaa17d71ffd8523d368247815fb47fa0fa6334e99b, 182c36a0398009d6b6e58301e13c805cd5acfc37412f55cadee180e90b27afdc, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032
diff --git a/test/data/gt_production.csv b/test/data/gt_production.csv
new file mode 100644
index 000000000..80569d037
--- /dev/null
+++ b/test/data/gt_production.csv
@@ -0,0 +1,97 @@
+pubkey, nonce, miningseed, score
+039cc1f1560aa96daa994a2b296f22d7f2fc9503ce95321d0b8193079e5f93dc, bee1e55fc2c967601827dd80f09a6b0a47ceab37b920b57efbd23c8fdc49e38f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4254
+9b3d041660f08f692ae1005118f85c35bec491c33b51c94ef126663f4402fb1e, 1a314eb596f55aa79e7f61334072380042ea4e351ca59dc09f536f8224a39afa, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4787
+5ddde3533e040840b737304dd92b4b48d62ab209419a8bf4506e7b0497e9c614, 419569517e0908f9bae5b07159d322204053afd4564b4079fe160f775e214993, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4329
+66950904cb50245726a50f9039e2d384bb878ce3e39e0575f64eb61bf9963325, 158191c4be8a6382d9eff13a448550c7742568909abc76d8213c958e85bb02f5, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4021
+beb0cbbaab3cc54e87e4b6ed4a5f6c4af2fa4f452a02252a8ec910c43fb71a43, 96725650ea0a2052e3175c0a1047f51fcc7031f1455fe168630f0ecbfce1a180, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4642
+c7f5024050c450389d341eabcea04b16e79b7a51f18bcb59284eaf40000cb7a6, 77b3965cec126453cfeb23b5b44f20d652058f7726f0e43e8c265eafba6e8418, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4138
+a6f49eb0c3fd08eeb56b4ab83bbfb0b415b02233101baff5eec26bb2d1455e34, ac8db67e608af98f13f70b611c71f527dbb626afe41128cc6dbb27da921a6db5, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4323
+5c49beb4e29c921d8fad67d9b1e09fd693254f9275dfd77ded5a66acf721b4fd, 5e80e9f095fadb379b12130c4796f74ec967fc526b1face4af813a13b029fcd3, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4596
+d8470f83e18ebd6bd221423f03b0e17a87872f6ef25ca8909f28078a6c300ff0, 7989ab5f03db1a93b51a8629ac6d569ff0299ee1744ddce813013e9a64ff6497, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4254
+4c83417da5f166accd29107292d5c0b7502811b4e0eef6d252c01474dce157bb, 59245fdd6fb52df651c4b3228596175e327aaaf4f4405cd37bf84ac1a9e51747, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4476
+aee5c8edcf3dfdab7ec84b7c580be8a64c0b902be6b365fe916733f05e238ac2, 56dddea0c9052587a47db7f5deaa16727663725edcbd0c5d78f9b1193bef12da, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4179
+f2112f60356004b9fe52840e06e9047d2020ba228b7e1987663491a7d90e3e85, 4893d1aeb7591ce464e26ed43e34501acf450a6d6ff3a4dbb85a3920d9011308, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4227
+050a8c2acdcc99c71b1f6e04f3785291f2980fcfbc0a2355a200eec5287a18d8, cd2744b32506fc199fb1aaf0b9edaa553d05f0f4c66e9f7d2e47140fbc7973e6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4112
+5bb6da0fd6e14135dcaee492185a223c5495936b29cabecf415094e1cf0b64a4, 1b169a9f717bd61508b009a9e149c3bf3bf192acbc2a11af6da31214ca0c1be2, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4367
+ce103ac663da7295f0013f33b1eadb5295e32dbf8392f67b8085caca9d8291da, 44fdb9bc59d865bd9d87f049ea4a7a702d1a0fca20cddb7172bb10d7c17dcb06, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4373
+da39c1bc6effaa3edb88cc805e6e7bd7aad1726ade6b0dd38d54b52957f14125, 50ee325b50af29c82a0ac06b3ec85885f9e36e08f1342a938acfd9a002ce524e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4538
+2f2a57841de709ea79ec678d1848a2dc13cb1d96d1d029278181e925ce6ce281, 29b9f9db6a5001480686637bb3f187868d23a8881b32482000a4e3b217074d54, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5044
+7bf6c1e4d203b633f0f5f5421d2d16e66db924cccc51092f1e89ededa54ad322, 490f7927f8f91dbeab6434e4f0ca660642d9157267613ded72a8315df821d838, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4272
+a6f0ae22305a9a0336a2c7d97e1568697cdb62e1aedfc9874a1ac1915b4e3c67, 332f47bf272e90de5befc330c0ae65b88a981c6c28819c9df8bbc7fccd7176c8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 5126
+1f710bfadc49f5a1bed75fb10054979a3b4856a48e3b75184f7b056d3a23ac41, 6d5adaad8e0ea9208390e5649d2e0c6365ee84cc1611ae5f765424d7992b8322, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4227
+36f621af4c726d3e5bc3cce3d26a4418c2272024fd8177110a6a64fcc9071afd, 6fa998c1ee8efba299b8b51e865a13b939df3189d94c0b939f86c8b2d9fb7893, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4002
+8117882a9cbfce4ca0df28f72a818773ba80377417e8c73d2c0a56e8c6e4ae22, c762e691c4918cc826e72b5277fa2c11f4ff9cb2a01efefabe6ab8f6c8c5be3e, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4124
+a7a69484aeb270fda2ce28d9b6d4ef6f1e73edde9adea5404f81b7ae265a24bc, 4b18c6bf59aeaf1f96a11956eba1c2ac7bbddbed8e0f12ec5a94dcc8107a5cb6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4500
+a84e3c009575dd77299c18d600d0252f9d0480d34c5da189a7625a0274f83cc8, e0c96c11ba7a7137aa7fe213dd34dee838a598acf0d76776e5c3778307c1e631, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4334
+dc100c8d058c0cf6df88dbf4fe01f6023b173be3bedd6b86dee4b8eecc72e058, 179fdf1f611d6c27a08eb533b826d169a0bd3d33e6c329ded1eee76fcefcd2dd, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4259
+087fa2a7ada94299393d118b19c6360699845db46f559eba3ee5b8bc494f2a8f, 0f200d7799483ecf52a4af5c3a69111fd29a14bd7a03a82f9f39f1913b29e5f8, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4347
+8f9069a8eb962dc47afac9ea7d4a77d7ab801c3e932183f9dcda9c8cd7fdaf9b, ab4f54740589f62c4e35c2b7a2508a3f7fb23872afd62771e57deea64bc540d9, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4477
+ecbce54faf01fbf9015cf9b19be9bbf42e9530be81fe09fc92c666eace8bfc15, 86acf64f19dc27151febf20c2549d49b05f4dc87290e108e935ebd6d49950cee, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4513
+36e28fe48780115b126255d893208b16f04505687e9aafd17ba73dbe543a8319, ab908ba2a064b7381f26c7a02cfaa2d8b7689d5f24877aaad99e25e0714ca19b, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4711
+ee4fca45aa951e2a2c0e535fddda71cd1c0722f1b30bf74416d9ac6df8787e52, b53606d34971e204c260e1c5c2bcb88450f429b76340d0860046bf2a67cc359f, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4634
+25b3ace18ee066c5cb561d32bfc89f69f20199741519d148a1d00312191e1f0f, 06ab93f88f0fe2aaa54efe1abf60cceef2b21d15c378d66096398c8b82947c7d, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4032
+a7149ab5f6c9f3ec2dc997564839a050de78dcf54e255d0019b4d81c3675bb56, accab93bf12ddf0a8b2ee43bcb94fe01d265eacd6df6aa8bcf133a5d359a79f6, 8b12add89bc264e01038b61b784e0778a03cd3025e1faeedda6598f197ceccc0, 4964
+d4aaeaf020007d1349590d77e2f779ad48f9a6cf720d04ca6a65fbbad81dc050, 7a40d7032c899ac35f224bd0a7be2c12244e9b2b4a32852b2ec880d526929ece, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 5357
+5fd98a26b2bcd498daef005d5035336777164fd594690d083ffc07c2750a00bf, f8c4c01377b01fb07d914d37301d2eee2349e015128adac766c676913f7c056e, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4538
+acc5d85d4bdb5d3e63879ee9975db8dd89474eaa143fd3a3b0dd9b406e6fb19a, 2097e387f6f27d6f630532f3407189adfebfae9bfe4308f0a4a718b72756cb64, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4251
+5329b924cde1d5be74ae7b98fd5dcd6c6cfc497abaabcfd13ba6ad76b3e80a73, 409da8d39041e54f20851bc0ae095cb8b80ba393af8531ea6f1977e14e900bca, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4435
+bd50121a1a9982b2e6f1becff5beced13d3aeb3b8292b6d64eaa641a85c6af36, e2c0de9cf3488fb4db674d9b6bdb7261044556a5ff21d66781c975316624ce8b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4064
+f09fa2dd45236ebd36dfca4443787119dcefdd152723cf5f4346f16c33611ca4, ab4f459cae0455dcee65258b8c8340e3429cb43d826001a52542b0b10b2d3051, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4618
+5df6390b68e67e1cff09fea890e14b204f8b5e9ef847b14a05b7b32f83576c73, d2c779399ff772a44024e4991f32463193e7d21e958b926ead5186fad44d276e, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4658
+0c5c814529d514ed2084f87554a1278e303aa627c720c21502509e6453b46e75, 4d2317b4d7645a12954929dd48e1541ab6c333eeb590fed06bc795876d399b86, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4805
+1ec0460a6d6ecba87d61bde51c2b27c758cefecc568cd807b5d34c51d4d9f427, d0aaeea09c9dee00f7062fabcf47fd61f2c288b80295a820ec4222a18c2f67cd, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 5385
+e91795a4cc4ea7a134da695bd2ca1638e3510d2ed21a6207c34b25c920925556, 2cfdfbd4047379ca02f323115287c8132cbd1ab84ae94bb4142147ccfaabcde9, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4104
+547ec5ed8156d06f369b8642d81f542aeb06d8be0fddfb82d499722def67ab8d, 58315e20f5e05abcbb1c225435e985dc08edb01a51bb472e5a2caa1ebea54636, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4293
+d8e5cac99aa0e4d53d13a2a5efcb44533def7116cd8cab23b3e8ffe2e6e36033, db5d8b11d57d382bb4a174aeaeea1934089ca1ce87f695b02e969aa2d804323b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4490
+11e6f5455923f425403174d26957f3c4343082676f2786a504b175a37e095257, 3d2b359ee811655af3afa43d4b6bcef022d9b086bf7b90196f8d2fd319cef2e9, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4101
+3c5904c5eef6ae6bd1c5f04b329bf8263f5de55ed27b416caec5d1c7ba8a1563, 4595621029e319dd6a712f40b4694bb41a198b97fcb5e1431e36b43d2a49f70c, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4229
+bd57f4064ecfd309f405a10536844d1ba5ee33ef367100f8a7bb8c0d30564c09, 3e3fe3eb8a8e57fd02bf0aed9362a8fa922792c23466f23910a07f5ecd6a5891, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4399
+8cd870e8a74f72c9293b449aff6f2703eee771a2e804ab3b0e85e90110a70ebe, e4043488f279cd9c732d5074e615a25c2684e84c82d86c742627ea7a5eacca33, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4427
+2aad58dee6a39af963379f65fe80379c9787fa0ef39ac6cfca83883b39d88f80, fe7024fbe4ed4989b6004ecc78ef92db203631b606d3bcc0266406bc2dc2c95d, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4211
+2944a74d098bc8cb2c49c7058fa5c1749b6b0762e7953d2909d081a621322bc9, a4251d12a25da7049ac54781cfcf729548fba470abd9aab5071854aa33651f48, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4392
+59423970ab4e01dcd75b21c9bb28039cc754cbd9504f1baf7690ad67f653d9bc, 8cb0ca3bc6616437eac5c040fcbc82de45a9374a686c68be73e1ece507fbe7a2, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4234
+4111344f7654a20f63f205251e233949ced281978307fe80f5994101ba30fff5, e0cb33733e52c5cfe2f342b834e6be21a4c8ba142184a5a712fbe01d2f321ca0, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4192
+cb2e9d7b14e92a7acd3606295181a85b3a3834812c85a58c5d8e8fb966d52e80, ffe82f2f47efff7e3c7c1f8489126be98246f789a0d5cff4e62e251d7e9637b5, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4246
+6acfa4e805379c4ad2c8fed920eb94bea5c66753f76c9e8903cb9f061b9a8568, 3291da84832c5e9fd2ec14ab4bc3852a9f501635962490595f92fccdb66104aa, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4277
+5950a4ccc1ee66af76e4ddec06caa7a0992cb2f59b7fe0468edd64f7bf7ac3e5, 8f6afeae5b25d78e7a9105fe200525e0012cd20ccf172dd0f610dbf804c1d731, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4140
+971e704c6ef92be3bedbdf77b8829804ee21011c6a2a3e79a68b1d4ad1cf826f, 19be39d709b2b1cd553f4ade9553d982a5ebc2028c7d61fa2693ce7c701ccf2b, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4122
+95ebe78a1bec9a12aa57462a7818c047b9cae0376666d0640905e81ff6e9e356, 7a418455ac94a63d6726f6fdecbf32d9426dd990ba0548e31e710703639980f2, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4418
+8782eea2fe0934ef90467ece2754218f16e02f731277ec7f244217f0ee0bb764, 7904e219e0265db109364c873d81f45e7e7522e881f9646faeba35fcc49985af, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4189
+7097a76021713afceeaa610bb79cffa218a8f6235ea1aefe078e957210170a47, 4068216c2a24e21043646f0f6b411628892c9350517451f528e632dc757ddabd, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4174
+e2c85076c6767b991f71921fa6e3bdd669a3e1e739385b4f0330250c828bf8ec, 017471a30b0129216db7a28a5da7dd2b16c4bc856c85d5135985119e4a99e6d3, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4336
+983e19c755f902f0026754de7fc89dbfba6ccc17f161a6f69c94e27ae5f3b8f4, 63301609c9ae6610af2aa1282c70531380a6d90e2b87d36ced6a84108f64755f, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4686
+66f9328e9c1a79595d52798a25d3459552afa6af9aa52ceb7ec76f38a8f3bd39, 0b71ad39ef27577b06bceab5f264201b2e8087143e0df90c69b0e206042470f1, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4307
+befe30abf1c5e24965d39b7ff3f2fc33fc4698be8a33c87aa4f8536ddb8a1996, ad87664522487ee7986f5ce7c26ff962b5feba171551b15b047d9a7053dedca5, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4235
+0363966c6d6ea832ddb1df165841a3479df9ec8d2ce0cea8a9400360f3cecc34, 391a80f81e72894605743026c249995103834322ce94fd953107a781583c2f23, 25b115eca9f2c1c0b3ada40faa46c248f2919d8e5f6e7fa2866d4d96ca1c0eaf, 4472
+2cfd43630593e07902ff52948ce387aaf0e2bb0e11952f8a0c91a33672203bcc, 157ebdf68c866bef7f93adafa1490ce1c15c3b951f23461f5327b53b3546726a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4874
+f99a06858d3d9dd741b0d1ce08556aab5c4b383b8199920d776daea35634b1b6, 494feb7026395e0ce15b603a2e424b79cea1d46da11fead0d84c29cae57299a5, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4657
+09bf69a84c357ff346577e5d17748c2b3d5b3ed84e9329759b131c2b94e7f4b5, 108075e0dfc82821119e0f323ab789ef809dc39b203db23ee3242f3a42ec0a80, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4728
+7ba937604ac1bdd1cdda1954b6b6a93eda265f468a49f4941f8de0980caed213, eaf8c0959fdd124b1756144818ba32a60f4fe12454bc342e25a74b13b745241a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4274
+060f7e1a8c8cb6f938b3116ff25884b6eb87fafe9ec957d07ca28ee58fea68ee, fdead1f49e206d1892dc77b38a8b0fc8a286d01683ee994f7fe591c6f66b2bdd, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4212
+377741f3abd82ac5b90df9edf3d59ca9ab1fea5376dadb40242c87f55a6c287a, 9669b5d4bf3ef9184daac77ab44f0e3ce4e46752e2a7fd68ebecfbcf6109fe3a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5152
+7a4fcbb032e872b47f2a33fbb2d27fa06965165a91ce284a8d29a43bb4d118aa, edd6bd4e7bff26c7e441011422fcfe753bded55005a5221046884efb2afafaa9, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4144
+6ceb9db060f76ff7a170037cf3fb0413f3bcb5b2cbdf7f42106d30fe6516be95, 9c83bb2e063e73e7338944f8babdfd18d01ed1d13c9baa43fa7489fb52f176c2, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 3956
+43e55c8788cad0c02cdf142a7f7e4683b332ed6a8efcc831c580aab3e9a1b27e, d9990b01cbe2746aafedc17d14f5a84675938f952eaeee0b3b5764e4bf054074, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4236
+cd0e79b31ed4e699811ca0fcbaf81cb3b0e08188acfc38b6a98e3f6d077e6fe7, 3625fb00a429fd89c4ce1c4f92303cececfbccfd7bc10b1f00d98e8e8540fbd6, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4421
+61217450605f85dfd56bb5c1c046db8018bec5c533f3d5dd92732ffdf8786e60, a165d267e0fa10de03d6149b6853930ba3f9a244f3dd07a66b202f064535dc6a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4330
+c7f29ca4e434a0283049a26b8dba34072bf7a90e2116a5374171b0137789fa34, cce63e8a939e4dbbcff17212af5f390f7b5d5c7ac7c1c4bdc9db353b324b092c, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5074
+19c0127c42618672cda53ef8998cf7a79bce5eda48aba533c4c157db73bf1430, 63f09dee35a7ad227266f20295f406311a3563765e51dcab2135f589127c5750, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4916
+93970bbe1875f9bd9766a225f95aa3d3e748ae4ff7a629e8d8067582ae6874be, 4e9fc8003d5384be5dde14ed04c9bb256dfc34882ae7c51873a74aba28e300c5, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4419
+6b17204cc22a12de5171f59978a5598908a3778e9d635e925190e312f27a5104, eb631efad97aa19034488b9b19fd26ac670fb584ed0f7eeb0bff2d4f1e29f0c4, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4692
+d0d2996accf533fac438f273ab65edc50eb5a756a727fa9c51308f8b8608583a, fd0ad8a263a4cae7cc1cba87fa8c5af6b0c74694cf6dcc9c56e4867d1467b53e, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4544
+f075dc821b7f48bb638dd898b9c1336abe8ac4b8ba5c964b5900c8b26fb89223, 88ed66223b41c277d464e33e1eca9e022df91b476a316050c9fe296c2d27a55a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4127
+c14ff23a8dd058c87e49edb5b6e0433e24238911ec1c7c179cd8338ad3f1eea5, c35e26a31a309a160cd348f3540aef3277f3ca07eb869c59634b6986f44ae8d0, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4488
+7baf3e8717e443d1f3a2459c6e4368b79a6233f75e236aa6367b5eaaa2deb8cc, 853d8bc62f96344aed6f69a683ff7bb15f82839fecadbbad8b2eeda3713b73d6, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 5043
+aaab6229bd572c77a7a89b3b9a7bd17cfd085528ac5471fc358b1a4c6d011c4a, 1276013c419de03614ac9da61dc28c732b5d706529e5492caab405db613d6340, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4216
+f73ab8675d3eef76ffd330ad2ef8c2dbdb2f5fe8e76bb7c5458639f2e2cdb7b4, 5c3dba3a97e7cf0d9b92e5cbb8228d5b6ea3f9dc54fba6c288e6181b03ba369f, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4014
+adede9708eecf496012fb63c6dba5926c896d077ff1bd2e8a911b4eced495d82, 66cbfcfbd434f0a4c066c6e42c54c8a5ff544eb81834c14781850421d3b87d28, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4549
+0c5a86ecee90b2275137aead69f1005ef9f1edc420845fa667ea020f98c8d064, b033d35f89a05b58388d4cfd3f2cb855ae4a3925353926b6f267a06d63fc9420, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4144
+f2ce52d93723e7c386d4604876a3c3ae7a06a295411ea2e3719f0d5b5e2a5e44, f3b1935402df86ee59dd2913a11fb327451542645a62bb8e5217a459e242e820, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4309
+f067e50164c5eab1194c3b77cc119365c01ec9501a7adf5becc7804c46695dff, f342f24d14e52fb7441f409386ab457e353caa06ee395beaa25e27be5e1a7c7b, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4517
+7a33c852aef3009f1a8096124c4e166115fd14c1619bf8a1275b94292787c274, 1f704e695d2b807cd9b3f877ab12fa6d5cabe77127a589fb2ae6df94470e7717, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4352
+34cd3c37f0cbda82af0b62b0b3c75887a1332c5a855218c75fe47e2d6bc6aeec, 35df8cd36c809e50e74990932a316fe5ecefcbb19fe3f4a8d027fff912202998, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4042
+dc679985b8b7d22ca9dcf52c947f826d9cf6f36582b70f7282ac1b67a0c46ae8, 80c2186977f4180ad43db66bb7d5596d7e42410dbaf7e84734b71d589f46724e, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4054
+17f3b29ae0d930361208d16169894ee4ed9b5846bd01147d0c7bcf05ae7d25e4, 5ef965812a3918a2dec24cedfb23a306a0305e40e52d9b38ff00b2a0d2ebe616, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4480
+836eb6226f84ecc4a270ea559f2b3418cdc0bb9eb66a26bcceaa7e45c802cdee, 9a7512dc2a55c148ee1900e85f9bec223bdd73ba25b650873433164d196f2349, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4849
+6e8090bf760bbd8dd5146ebe3420223b8be6b30b65e714220c27d1c2ab62423e, 29ad5c73e1f39167a370020bc2ed3af84e640bfcde167a6981f0114723fd7ac9, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4296
+9c256502b40844cd9f2b11ed9eb5472441731b66797542837dbc68ee2524cb98, 5627705fd251600ddff839d02077e646f85dbb08fc08947e9098d0e2c289022a, 96d417dd9476db171dfdb190e1e2890fcc8ac78965ad0a65d7c2d2705d49d5d5, 4454
diff --git a/test/score.cpp b/test/score.cpp
index 451edc2a0..2fe37b43c 100644
--- a/test/score.cpp
+++ b/test/score.cpp
@@ -7,6 +7,7 @@
#include "../src/public_settings.h"
#include "../src/mining/score_bpp9000.h"
#include "../src/mining/task_file.h"
+#include "../src/score.h"
#include "score_bpp9000_reference.h"
#include "score_params.h"
@@ -23,7 +24,9 @@
#include
#include
#include
+#include
#include
+#include
using namespace score_params;
using namespace test_utils;
@@ -32,13 +35,17 @@ static const std::string TASK_FILE_NAME = "data/example_task_bpp9000.bin";
static const std::string SAMPLES_FILE_NAME = "data/samples_bpp9000.csv";
static const std::string SCORES_FILE_NAME = "data/scores_bpp9000.csv";
+static const std::string PRODUCTION_TASK_FILE_NAME = "data/bpp9000.task";
+static const std::string PRODUCTION_FILE_NAME = "data/gt_production.csv";
+static const std::string PRODUCTION_ANT_FILE_NAME = "data/gt_ant_production.csv";
+
// true = ALSO run the engine-vs-reference cross-check on random tasks, for isolating a divergence.
static bool gCompareReference = false;
// Samples run per config
static constexpr unsigned long long TEST_NUMBER_OF_SAMPLES = 32;
-// Worker threads for the parallel path; effective count = min(this, hardware_concurrency, numSamples).
-static constexpr unsigned int TEST_NUMBER_OF_THREADS = 0;
+// Worker threads for the parallel path; min with hardware_concurrency and numSamples. 0 falls back to 1 (serial).
+static constexpr unsigned int TEST_NUMBER_OF_THREADS = 4;
// Samples and worker threads for the Bpp9000Profile timing run.
static constexpr unsigned long long PROFILING_NUMBER_OF_SAMPLES = 48;
@@ -196,7 +203,8 @@ static unsigned int workerThreadCount(unsigned long long numSamples)
{
hw = 1;
}
- unsigned int chosen = (hw < TEST_NUMBER_OF_THREADS) ? hw : TEST_NUMBER_OF_THREADS; // min(hardware_concurrency, chosen)
+ unsigned int requested = (TEST_NUMBER_OF_THREADS == 0) ? 1u : TEST_NUMBER_OF_THREADS; // 0 falls back to 1 (serial)
+ unsigned int chosen = (hw < requested) ? hw : requested; // min(hardware_concurrency, requested)
return (unsigned int)((numSamples < (unsigned long long)chosen) ? numSamples : (unsigned long long)chosen); // no more than one thread per sample
}
@@ -378,6 +386,179 @@ TEST(TestQubicScoreFunction, Bpp9000Regression)
runRegression(seeds, pubkeys, nonces, taskBytes, golden);
}
+TEST(TestQubicScoreFunction, Bpp9000ProductionRegression)
+{
+ auto rows = readCSV(PRODUCTION_FILE_NAME);
+ ASSERT_GT(rows.size(), 1u) << "missing/empty " << PRODUCTION_FILE_NAME;
+
+ std::vector pubkeys;
+ std::vector nonces;
+ std::vector golden;
+ std::vector uniqueSeeds; // distinct mining seeds -> one pool each
+ std::vector poolIndex; // per row: index into uniqueSeeds
+ for (unsigned long long i = 1; i < rows.size(); ++i)
+ {
+ pubkeys.push_back(hexTo32Bytes(trim(rows[i][0]), 32));
+ nonces.push_back(hexTo32Bytes(trim(rows[i][1]), 32));
+ const m256i seed = hexTo32Bytes(trim(rows[i][2]), 32);
+ golden.push_back((unsigned int)std::stoul(trim(rows[i][3])));
+
+ unsigned int idx = (unsigned int)uniqueSeeds.size();
+ for (unsigned int k = 0; k < uniqueSeeds.size(); ++k)
+ {
+ if (memcmp(uniqueSeeds[k].m256i_u8, seed.m256i_u8, 32) == 0)
+ {
+ idx = k;
+ break;
+ }
+ }
+ if (idx == uniqueSeeds.size())
+ {
+ uniqueSeeds.push_back(seed);
+ }
+ poolIndex.push_back(idx);
+ }
+ ASSERT_FALSE(pubkeys.empty());
+
+ std::vector> pools(uniqueSeeds.size());
+ for (size_t k = 0; k < uniqueSeeds.size(); ++k)
+ {
+ generatePool(uniqueSeeds[k], pools[k]);
+ }
+
+ // The production task
+ auto taskBytes = readBinaryFile(PRODUCTION_TASK_FILE_NAME);
+ ASSERT_GT(taskBytes.size(), sizeof(score_task_file::TaskFileHeader)) << "missing/short " << PRODUCTION_TASK_FILE_NAME;
+ const TaskBlocks tb = taskSubview(taskBytes);
+
+ runWorkers(workerThreadCount(pubkeys.size()), [&](unsigned int threadIdx, unsigned int numThreads)
+ {
+ auto engine = makeEngine(tb.topo, tb.data);
+ if (!engine)
+ {
+ return;
+ }
+ for (unsigned long long s = threadIdx; s < pubkeys.size(); s += numThreads)
+ {
+ const unsigned int score = engine->computeScore(pubkeys[s].m256i_u8, nonces[s].m256i_u8, pools[poolIndex[s]].data());
+ EXPECT_EQ(score, golden[s]) << "gt_production row " << s;
+ }
+ });
+}
+
+// Ant-colony score seam (deriveRootANN + computeScoreFromParent)
+TEST(TestQubicScoreFunction, Bpp9000AntColonyRegression)
+{
+ // Group by chain each chain is a lineage - level 0 extends the derived root, level i extends level i-1's bestANN.
+ auto rows = readCSV(PRODUCTION_ANT_FILE_NAME);
+ ASSERT_GT(rows.size(), 1u) << "missing/empty " << PRODUCTION_ANT_FILE_NAME;
+
+ struct AntNode
+ {
+ m256i nonce;
+ m256i anchor;
+ unsigned int score;
+ };
+ struct AntChain
+ {
+ m256i pubkey;
+ unsigned int poolIndex;
+ std::vector nodes; // indexed by depth
+ };
+ std::vector chains;
+ std::vector chainIds; // chain id per slot, first-seen order
+ std::vector uniqueSeeds;
+
+ for (unsigned long long i = 1; i < rows.size(); ++i)
+ {
+ const int chainId = std::stoi(trim(rows[i][0]));
+ const int depth = std::stoi(trim(rows[i][1]));
+ const m256i pubkey = hexTo32Bytes(trim(rows[i][2]), 32);
+ const m256i seed = hexTo32Bytes(trim(rows[i][5]), 32);
+
+ unsigned int sidx = (unsigned int)uniqueSeeds.size();
+ for (unsigned int k = 0; k < uniqueSeeds.size(); ++k)
+ {
+ if (memcmp(uniqueSeeds[k].m256i_u8, seed.m256i_u8, 32) == 0)
+ {
+ sidx = k;
+ break;
+ }
+ }
+ if (sidx == uniqueSeeds.size())
+ {
+ uniqueSeeds.push_back(seed);
+ }
+
+ size_t cidx = chains.size();
+ for (size_t k = 0; k < chainIds.size(); ++k)
+ {
+ if (chainIds[k] == chainId)
+ {
+ cidx = k;
+ break;
+ }
+ }
+ if (cidx == chains.size())
+ {
+ chainIds.push_back(chainId);
+ AntChain created;
+ created.pubkey = pubkey;
+ created.poolIndex = sidx;
+ chains.push_back(created);
+ }
+
+ AntNode node;
+ node.nonce = hexTo32Bytes(trim(rows[i][3]), 32);
+ node.anchor = hexTo32Bytes(trim(rows[i][4]), 32);
+ node.score = (unsigned int)std::stoul(trim(rows[i][6]));
+
+ AntChain& chain = chains[cidx];
+ if ((size_t)depth >= chain.nodes.size())
+ {
+ chain.nodes.resize((size_t)depth + 1);
+ }
+ chain.nodes[(size_t)depth] = node;
+ }
+ ASSERT_FALSE(chains.empty());
+
+ std::vector> pools(uniqueSeeds.size());
+ for (size_t k = 0; k < uniqueSeeds.size(); ++k)
+ {
+ generatePool(uniqueSeeds[k], pools[k]);
+ }
+
+ auto taskBytes = readBinaryFile(PRODUCTION_TASK_FILE_NAME);
+ ASSERT_GT(taskBytes.size(), sizeof(score_task_file::TaskFileHeader)) << "missing/short " << PRODUCTION_TASK_FILE_NAME;
+ const TaskBlocks tb = taskSubview(taskBytes);
+
+ // Thread across chains; a chain is sequential (each node's bestANN feeds the next depth's parent).
+ runWorkers(workerThreadCount(chains.size()), [&](unsigned int threadIdx, unsigned int numThreads)
+ {
+ auto engine = makeEngine(tb.topo, tb.data);
+ if (!engine)
+ {
+ return;
+ }
+ for (size_t ci = threadIdx; ci < chains.size(); ci += numThreads)
+ {
+ const AntChain& chain = chains[ci];
+ const unsigned char* pool = pools[chain.poolIndex].data();
+
+ score_engine::ScoreBpp9000::ANN parent;
+ engine->deriveRootANN(chain.pubkey.m256i_u8, pool, parent); // depth 0's parent = the derived root
+ for (size_t d = 0; d < chain.nodes.size(); ++d)
+ {
+ const AntNode& node = chain.nodes[d];
+ const unsigned int score = engine->computeScoreFromParent(
+ parent, chain.pubkey.m256i_u8, node.nonce.m256i_u8, node.anchor.m256i_u8, pool);
+ EXPECT_EQ(score, node.score) << "gt_ant chain " << ci << " depth " << d;
+ engine->getBestANN(parent); // this node becomes the next depth's parent
+ }
+ }
+ });
+}
+
// TestBpp9000, internal score vs the score reference from Qiner
TEST(TestQubicScoreFunction, Bpp9000EngineVsReference)
{
@@ -506,5 +687,678 @@ TEST(TestQubicScoreFunction, Bpp9000Profile)
}
#endif
+// =============================================================================
+// Ant-colony related
+
+namespace
+{
+using AntCfg = ProductionConfig;
+using AntEngine = score_engine::ScoreBpp9000;
+
+// Pool + synthetic task + loaded engine. Every ant test starts from one of these.
+template
+struct AntFixtureT
+{
+ std::vector pool;
+ std::vector taskBytes;
+ std::unique_ptr> engine;
+};
+using AntFixture = AntFixtureT;
+
+template
+static bool makeAntFixtureT(AntFixtureT& f)
+{
+ std::vector seeds;
+ std::vector pubkeys;
+ std::vector nonces;
+ loadSamples(seeds, pubkeys, nonces, 1);
+ if (seeds.empty())
+ {
+ ADD_FAILURE() << "missing/short " << SAMPLES_FILE_NAME;
+ return false;
+ }
+ generatePool(seeds[0], f.pool);
+
+ f.taskBytes = readBinaryFile(TASK_FILE_NAME);
+ if (f.taskBytes.size() <= sizeof(score_task_file::TaskFileHeader))
+ {
+ ADD_FAILURE() << "missing/short " << TASK_FILE_NAME;
+ return false;
+ }
+ const TaskBlocks tb = taskSubview(f.taskBytes);
+ f.engine = makeEngine(tb.topo, tb.data);
+ return f.engine != nullptr;
+}
+
+static bool makeAntFixture(AntFixture& f)
+{
+ return makeAntFixtureT(f);
+}
+
+static m256i makePubkey(unsigned char tag)
+{
+ m256i k = m256i::zero();
+ k.m256i_u8[0] = tag;
+ k.m256i_u8[31] = (unsigned char)(tag * 3 + 1);
+ return k;
+}
+
+// Canonical ant nonce: nonce[0] selects bpp9000, nonce[1] = L, nonce[2] = K, rest is the walk seed.
+static m256i makeAntNonce(unsigned char L, unsigned char K, unsigned char tag)
+{
+ m256i n = m256i::zero();
+ n.m256i_u8[0] = (unsigned char)score_engine::AlgoType::Bpp9000;
+ n.m256i_u8[1] = L;
+ n.m256i_u8[2] = K;
+ n.m256i_u8[3] = tag;
+ n.m256i_u8[17] = (unsigned char)(tag ^ 0x5A);
+ return n;
+}
+
+// Find a nonce whose walk from this parent actually improves on the parent's score. Needed only by
+// tests that compare two children of the SAME parent
+static bool findImprovingNonce(AntEngine& engine, const AntEngine::ANN& parent, const m256i& pk,
+ const m256i& anchor, const unsigned char* pool, m256i& outNonce)
+{
+ for (unsigned char tag = 1; tag <= 6; ++tag)
+ {
+ const m256i n = makeAntNonce(6, 5, (unsigned char)(50 + tag));
+ const unsigned int sc = engine.computeScoreFromParent(parent, pk.m256i_u8, n.m256i_u8,
+ anchor.m256i_u8, pool);
+ if (sc != score_engine::INVALID_SCORE_VALUE)
+ {
+ outNonce = n;
+ return true;
+ }
+ }
+ return false;
+}
+}
+
+// Make sure the score at the full computeScore flow have the same score with
+// the engine start directly from the best/final LUT
+TEST(TestQubicScoreAntColony, BestAnnReproducesReturnedScore)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(1);
+ const m256i nonce = makeAntNonce(3, 0, 11);
+
+ const unsigned int best = f.engine->computeScore(pk.m256i_u8, nonce.m256i_u8, f.pool.data());
+ // Re-score the LUT the walk kept, taken out and put back through the public form - this also
+ // exercises the compact/expand round trip the tree relies on.
+ AntEngine::ANN bestLut;
+ f.engine->getBestANN(bestLut);
+ f.engine->expand(bestLut, f.engine->currentANN);
+ EXPECT_EQ(f.engine->score(), best);
+}
+
+// Make sure the score at the full computeScoreFromParent flow have the same score with
+// the engine start directly from the best/final LUT
+TEST(TestQubicScoreAntColony, BestAnnReproducesScoreFromParent)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(2);
+ const m256i nonce = makeAntNonce(4, 2, 23);
+ const m256i anchor = makePubkey(9);
+
+ AntEngine::ANN root;
+ f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), root);
+
+ const unsigned int childScore = f.engine->computeScoreFromParent(
+ root, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data());
+
+ // Re-score the LUT the walk kept, taken out and put back through the public form - this also
+ // exercises the compact/expand round trip the tree relies on.
+ AntEngine::ANN bestLut;
+ f.engine->getBestANN(bestLut);
+ f.engine->expand(bestLut, f.engine->currentANN);
+ EXPECT_EQ(f.engine->score(), childScore);
+}
+
+// An ANN is exactly its LUT: no storage padding escapes the engine, so hashing or shipping one is
+// just sizeof(ANN) and a future change to lutStride cannot alter a digest or the wire format.
+TEST(TestQubicScoreAntColony, AnnCarriesOnlyTheLut)
+{
+ static_assert(sizeof(AntEngine::ANN) == AntCfg::populationThreshold * AntEngine::lutSize,
+ "ANN must be the LUT and nothing else");
+ static_assert(sizeof(AntEngine::ANN) < sizeof(AntEngine::PaddedLut),
+ "the working layout is the padded one, not the other way round");
+
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(3);
+ AntEngine::ANN root;
+ f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), root);
+
+ // Every byte handed out is a trit; nothing from the padded rows leaked in.
+ for (unsigned long long i = 0; i < sizeof(root.lut); ++i)
+ {
+ ASSERT_LT(root.lut[i], 3) << "byte " << i << " of the returned ANN is not a trit";
+ }
+
+ // Scribbling on the working layout's padding cannot change what comes out of it.
+ AntEngine::PaddedLut working;
+ f.engine->expand(root, working);
+ for (unsigned long long k = 0; k < AntEngine::maxNumberOfNeurons; ++k)
+ {
+ for (unsigned long long b = AntEngine::lutSize; b < AntEngine::lutStride; ++b)
+ {
+ working.lut[k * AntEngine::lutStride + b] = (unsigned char)(0xA5 + k + b);
+ }
+ }
+ AntEngine::ANN again;
+ f.engine->compact(working, again);
+ EXPECT_EQ(memcmp(&again, &root, sizeof(root)), 0) << "storage padding reached the ANN";
+}
+
+// expand/compact must be lossless, since every parent read from the tree goes through expand and
+// every child written back goes through compact.
+TEST(TestQubicScoreAntColony, AnnSurvivesExpandAndCompact)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ AntEngine::ANN original;
+ f.engine->deriveRootANN(makePubkey(44).m256i_u8, f.pool.data(), original);
+
+ AntEngine::PaddedLut working;
+ f.engine->expand(original, working);
+ AntEngine::ANN restored;
+ f.engine->compact(working, restored);
+
+ EXPECT_EQ(memcmp(&restored, &original, sizeof(original)), 0) << "expand/compact is not lossless";
+}
+
+TEST(TestQubicScoreAntColony, RootAnnIsDeterministicAndPerIdentity)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pkA = makePubkey(4);
+ const m256i pkB = makePubkey(5);
+
+ AntEngine::ANN a1;
+ AntEngine::ANN a2;
+ AntEngine::ANN b1;
+
+ f.engine->deriveRootANN(pkA.m256i_u8, f.pool.data(), a1);
+ // Deriving another identity's root overwrites initValue.lutInit, which is the state a1 came from.
+ f.engine->deriveRootANN(pkB.m256i_u8, f.pool.data(), b1);
+ f.engine->deriveRootANN(pkA.m256i_u8, f.pool.data(), a2);
+
+ EXPECT_EQ(memcmp(&a1, &a2, sizeof(a1)), 0) << "root depends on engine state";
+ EXPECT_NE(memcmp(&a1, &b1, sizeof(a1)), 0) << "two identities share a root";
+}
+
+// A child is a function of (parent, pubkey, nonce, anchor). Same inputs must give the same score AND
+// the same inherited LUT; a different parent must not give the same child.
+TEST(TestQubicScoreAntColony, ChildIsDeterministicAndInheritsParent)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(6);
+ const m256i nonce = makeAntNonce(5, 3, 41);
+ const m256i anchor = makePubkey(12);
+
+ AntEngine::ANN parentA;
+ AntEngine::ANN parentB;
+ f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parentA);
+ f.engine->deriveRootANN(makePubkey(7).m256i_u8, f.pool.data(), parentB);
+
+ const unsigned int s1 = f.engine->computeScoreFromParent(
+ parentA, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data());
+ AntEngine::ANN child1;
+ f.engine->getBestANN(child1);
+
+ const unsigned int s2 = f.engine->computeScoreFromParent(
+ parentA, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data());
+
+ EXPECT_EQ(s1, s2) << "same inputs gave different scores";
+ AntEngine::ANN child2;
+ f.engine->getBestANN(child2);
+ EXPECT_EQ(memcmp(&child1, &child2, sizeof(child1)), 0) << "same inputs gave a different child LUT";
+
+ f.engine->computeScoreFromParent(parentB, pk.m256i_u8, nonce.m256i_u8, anchor.m256i_u8, f.pool.data());
+ AntEngine::ANN child3;
+ f.engine->getBestANN(child3);
+ EXPECT_NE(memcmp(&child1, &child3, sizeof(child1)), 0) << "the parent LUT was not inherited";
+}
+
+// The anchor digest is part of the child's walk seed, so the same nonce on the same parent must not
+// produce the same child at a different anchor.
+TEST(TestQubicScoreAntColony, ChildDependsOnAnchorDigest)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(8);
+ const m256i anchorA = makePubkey(20);
+ const m256i anchorB = makePubkey(21);
+
+ AntEngine::ANN parent;
+ f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parent);
+
+ m256i nonce;
+ ASSERT_TRUE(findImprovingNonce(*f.engine, parent, pk, anchorA, f.pool.data(), nonce))
+ << "no nonce improved on this parent, so bestANN would not move and the comparison below "
+ "would be vacuous";
+
+ f.engine->computeScoreFromParent(parent, pk.m256i_u8, nonce.m256i_u8, anchorA.m256i_u8, f.pool.data());
+ AntEngine::ANN c1;
+ f.engine->getBestANN(c1);
+
+ f.engine->computeScoreFromParent(parent, pk.m256i_u8, nonce.m256i_u8, anchorB.m256i_u8, f.pool.data());
+ AntEngine::ANN c2;
+ f.engine->getBestANN(c2);
+
+ EXPECT_NE(memcmp(&c1, &c2, sizeof(c1)), 0) << "anchor digest does not reach the walk";
+}
+// Non-canonical nonces are refused by the scorer itself, so no caller can score first and check after.
+TEST(TestQubicScoreAntColony, NonCanonicalNonceIsRejected)
+{
+ AntFixture f;
+ ASSERT_TRUE(makeAntFixture(f));
+
+ const m256i pk = makePubkey(10);
+ const m256i anchor = makePubkey(30);
+ AntEngine::ANN parentA;
+ AntEngine::ANN parentB;
+ f.engine->deriveRootANN(pk.m256i_u8, f.pool.data(), parentA);
+ f.engine->deriveRootANN(makePubkey(11).m256i_u8, f.pool.data(), parentB);
+
+ constexpr unsigned char maxK = (unsigned char)AntCfg::numberOfMutations;
+ const m256i good = makeAntNonce(3, 5, 62);
+
+ // A rejected nonce and a timed-out walk
+ f.engine->computeScoreFromParent(parentA, pk.m256i_u8, good.m256i_u8, anchor.m256i_u8, f.pool.data());
+ AntEngine::ANN afterA;
+ f.engine->getBestANN(afterA);
+
+ // L below range, L above range, K above numberOfMutations, wrong algorithm slot.
+ static constexpr unsigned int numberOfBadNonces = 4;
+ m256i bad[numberOfBadNonces];
+ bad[0] = makeAntNonce(0, 0, 64);
+ bad[1] = makeAntNonce((unsigned char)(score_engine::MAX_LUT_ENTRIES_PER_STEP + 1), 0, 65);
+ bad[2] = makeAntNonce(3, (unsigned char)(maxK + 1), 66);
+ bad[3] = makeAntNonce(3, 0, 67);
+ bad[3].m256i_u8[0] = (unsigned char)score_engine::AlgoType::Neuraxon;
+
+ for (unsigned int i = 0; i < numberOfBadNonces; i++)
+ {
+ // The bad nonce is early rejected in computeScoreFromParent()
+ EXPECT_EQ(f.engine->computeScoreFromParent(parentB, pk.m256i_u8, bad[i].m256i_u8, anchor.m256i_u8, f.pool.data()),
+ score_engine::INVALID_SCORE_VALUE) << "non-canonical nonce " << i << " accepted";
+
+ AntEngine::ANN now;
+ f.engine->getBestANN(now);
+ EXPECT_EQ(memcmp(&now, &afterA, sizeof(now)), 0) << "rejected nonce " << i << " still ran the walk";
+ }
+
+ // Now after bad nonce, we feed good nonce, we expect this is ok
+ f.engine->computeScoreFromParent(parentB, pk.m256i_u8, good.m256i_u8, anchor.m256i_u8, f.pool.data());
+ AntEngine::ANN afterB;
+ f.engine->getBestANN(afterB);
+ EXPECT_NE(memcmp(&afterB, &afterA, sizeof(afterB)), 0) << "canonical nonce was not scored";
+}
+
+
+// L and K boundaries of the canonical rule, checked as a pure predicate so no walk is needed.
+TEST(TestQubicScoreAntColony, NonceCanonicalRuleBoundaries)
+{
+ using AntScorer = score_engine::ScoreBpp9000;
+ constexpr unsigned char maxL = (unsigned char)score_engine::MAX_LUT_ENTRIES_PER_STEP;
+ constexpr unsigned char maxK = (unsigned char)AntScorer::numberOfMutations;
+
+ EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(1, 0, 70).m256i_u8));
+ EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(maxL, 0, 71).m256i_u8));
+ EXPECT_TRUE(AntScorer::isCanonicalAntNonce(makeAntNonce(3, maxK, 72).m256i_u8));
+
+ EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce(0, 0, 73).m256i_u8));
+ EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce((unsigned char)(maxL + 1), 0, 74).m256i_u8));
+ EXPECT_FALSE(AntScorer::isCanonicalAntNonce(makeAntNonce(3, (unsigned char)(maxK + 1), 75).m256i_u8));
+}
+
+
+// ---------------------------------------------------------------------------
+// ScoreFunction task queue.
+
+typedef ScoreFunction<1> TaskQueueScoreFunction;
+
+static constexpr unsigned int TASK_QUEUE_PROBE_CAPACITY = 256;
+
+// What the work functions record, so a test can see which tasks ran and what they received.
+struct TaskQueueProbe
+{
+ std::atomic runCount[TASK_QUEUE_PROBE_CAPACITY];
+ std::atomic altRunCount;
+ std::atomic payloadMismatches;
+ std::atomic started;
+ std::atomic finished;
+
+ void reset()
+ {
+ for (unsigned int i = 0; i < TASK_QUEUE_PROBE_CAPACITY; i++)
+ {
+ runCount[i].store(0);
+ }
+ altRunCount.store(0);
+ payloadMismatches.store(0);
+ started.store(0);
+ finished.store(0);
+ }
+};
+
+struct TaskQueuePayload
+{
+ TaskQueueProbe* probe;
+ unsigned int id;
+ unsigned int patternSize;
+ unsigned char pattern[64];
+};
+static_assert(sizeof(TaskQueuePayload) <= TaskQueueScoreFunction::TASK_PAYLOAD_MAX,
+ "TaskQueuePayload must fit one queue slot");
+
+// Bigger than one slot, but starts with a valid payload so a wrongly accepted task records the run
+// instead of dereferencing garbage.
+struct TaskQueueOversizedPayload
+{
+ TaskQueuePayload base;
+ unsigned char extra[TaskQueueScoreFunction::TASK_PAYLOAD_MAX];
+};
+
+static TaskQueueProbe gTaskQueueProbe;
+static std::unique_ptr gTaskQueueOwner;
+static std::atomic gTaskQueueHelpersStop;
+
+static TaskQueuePayload makeTaskQueuePayload(unsigned int id, unsigned int patternSize = sizeof(TaskQueuePayload::pattern))
+{
+ TaskQueuePayload task;
+ setMem(&task, sizeof(task), 0);
+ task.probe = &gTaskQueueProbe;
+ task.id = id;
+ task.patternSize = patternSize;
+ // Fill the pattern with id+i, so it varies by task and by position
+ for (unsigned int i = 0; i < patternSize; i++)
+ {
+ task.pattern[i] = (unsigned char)(id + i);
+ }
+ return task;
+}
+
+// Records the run and checks the payload survived the copy into and out of the queue.
+static void countTaskRun(unsigned long long, void* payload)
+{
+ const TaskQueuePayload* task = (const TaskQueuePayload*)payload;
+ if (task->id >= TASK_QUEUE_PROBE_CAPACITY)
+ {
+ // Surfaces as a failed test rather than a write past runCount.
+ task->probe->payloadMismatches.fetch_add(1);
+ return;
+ }
+ if (task->patternSize > sizeof(task->pattern))
+ {
+ // A scalar that did not survive the copy is itself a mismatch, and it must not be trusted as
+ // the loop bound below.
+ task->probe->payloadMismatches.fetch_add(1);
+ return;
+ }
+ for (unsigned int i = 0; i < task->patternSize; i++)
+ {
+ if (task->pattern[i] != (unsigned char)(task->id + i))
+ {
+ task->probe->payloadMismatches.fetch_add(1);
+ break;
+ }
+ }
+ task->probe->runCount[task->id].fetch_add(1);
+}
+
+class TestQubicScoreTaskQueue : public ::testing::Test
+{
+protected:
+ void SetUp() override
+ {
+ if (gTaskQueueOwner.get() == nullptr)
+ {
+ gTaskQueueOwner.reset(new TaskQueueScoreFunction());
+ }
+ gTaskQueueOwner->resetTaskQueue();
+ gTaskQueueProbe.reset();
+ gTaskQueueHelpersStop.store(false);
+ }
+
+ TaskQueueScoreFunction& queue()
+ {
+ return *gTaskQueueOwner;
+ }
+};
+
+// Task run once. Normal case
+TEST_F(TestQubicScoreTaskQueue, EveryTaskRunsExactlyOnce)
+{
+ const unsigned int taskCount = TASK_QUEUE_PROBE_CAPACITY;
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task)));
+ }
+
+ // Try to process every task in queue until all done
+ queue().runUntilDone(0);
+
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ // Each task is expected run once
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i;
+ }
+}
+
+// Mixed mutiple size of tasks
+TEST_F(TestQubicScoreTaskQueue, PayloadArrivesIntact)
+{
+ // Bytes a task of this pattern length hands to addTask.
+ const auto taskQueuePayloadBytes = [](unsigned int patternSize) -> unsigned int
+ {
+ return (unsigned int)offsetof(TaskQueuePayload, pattern) + patternSize;
+ };
+
+ const unsigned int patternSizes[] = { 0, sizeof(TaskQueuePayload::pattern) };
+ const unsigned int sizeCount = (unsigned int)(sizeof(patternSizes) / sizeof(patternSizes[0]));
+ const unsigned int perSize = 4;
+
+ unsigned int id = 0;
+ for (unsigned int s = 0; s < sizeCount; s++)
+ {
+ for (unsigned int i = 0; i < perSize; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(id, patternSizes[s]);
+ EXPECT_TRUE(queue().addTask(countTaskRun, &task, taskQueuePayloadBytes(patternSizes[s])));
+ id++;
+ }
+ }
+
+ // Try to process every task in queue until all done
+ queue().runUntilDone(0);
+
+ EXPECT_EQ(gTaskQueueProbe.payloadMismatches.load(), 0u);
+ for (unsigned int i = 0; i < id; i++)
+ {
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i;
+ }
+}
+
+// A payload larger than one slot must be refused, not truncated into the slot or written past it.
+TEST_F(TestQubicScoreTaskQueue, OversizedPayloadIsRejected)
+{
+ TaskQueueOversizedPayload oversized;
+ setMem(&oversized, sizeof(oversized), 0);
+ oversized.base = makeTaskQueuePayload(0);
+
+ EXPECT_FALSE(queue().addTask(countTaskRun, &oversized, sizeof(oversized)));
+
+ // Nothing was queued, so the drain has nothing to run.
+ queue().runUntilDone(0);
+ EXPECT_EQ(gTaskQueueProbe.runCount[0].load(), 0u);
+}
+
+// The queue is bounded. Filling it until addTask refuses shows where the bound is, and that going
+// past it fails instead of writing off the end of the array.
+TEST_F(TestQubicScoreTaskQueue, QueueRejectsOverflow)
+{
+ unsigned long long accepted = 0;
+ for (unsigned long long i = 0; i < NUMBER_OF_TRANSACTIONS_PER_TICK + 16; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(0);
+ const bool added = queue().addTask(countTaskRun, &task, sizeof(task));
+ if (!added)
+ {
+ break;
+ }
+ accepted++;
+ }
+
+ EXPECT_EQ(accepted, NUMBER_OF_TRANSACTIONS_PER_TICK);
+}
+
+// The drain must return only after every task has finished, including the ones other threads picked
+// up. Returning once the last task was merely taken would leave work still running.
+TEST_F(TestQubicScoreTaskQueue, DrainWaitsForTasksRunningOnOtherThreads)
+{
+ // Stays in flight long enough that a drain returning on tasks taken, rather than tasks finished,
+ // would be visible.
+ const TaskQueueScoreFunction::WorkFunc slowTaskRun = [](unsigned long long, void* payload)
+ {
+ const TaskQueuePayload* task = (const TaskQueuePayload*)payload;
+ task->probe->started.fetch_add(1);
+ std::this_thread::sleep_for(std::chrono::milliseconds(2));
+ task->probe->finished.fetch_add(1);
+ };
+
+ const unsigned int taskCount = 64;
+ const unsigned int helperCount = 4;
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(slowTaskRun, &task, sizeof(task)));
+ }
+
+ // Create another threads for process some tasks in queues
+ std::vector helpers;
+ for (unsigned int t = 0; t < helperCount; t++)
+ {
+ const unsigned long long helperProcessorNumber = t + 1;
+ helpers.emplace_back([helperProcessorNumber]()
+ {
+ // What a request processor does: keep offering to run queued work until told to stop.
+ while (!gTaskQueueHelpersStop.load())
+ {
+ gTaskQueueOwner->tryProcessOneTask(helperProcessorNumber);
+ }
+ });
+ }
+
+ // Mark the task queue ready and process remained task
+ queue().runUntilDone(0);
+ const unsigned int finishedOnReturn = gTaskQueueProbe.finished.load();
+
+ gTaskQueueHelpersStop.store(true);
+ for (unsigned int t = 0; t < helperCount; t++)
+ {
+ helpers[t].join();
+ }
+
+ // Expect all task are done
+ EXPECT_EQ(finishedOnReturn, taskCount);
+ EXPECT_EQ(gTaskQueueProbe.started.load(), taskCount);
+}
+
+// Tasks are queued before the drain opens the queue. Until it does, a helper must pick up nothing, so
+// a half-built batch is never started.
+TEST_F(TestQubicScoreTaskQueue, ClosedQueueHandsOutNothing)
+{
+ const unsigned int taskCount = 8;
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task)));
+ }
+
+ // Try to run many task but no thing run because the queue is not ready
+ for (unsigned int i = 0; i < 32; i++)
+ {
+ queue().tryProcessOneTask(0);
+ }
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 0u) << "task " << i << " ran before the drain";
+ }
+
+ // Process all items
+ queue().runUntilDone(0);
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i;
+ }
+}
+
+// Every tick resets the queue and refills it, so a second batch must behave like the first. It will
+// not if reset leaves any of the three counters behind.
+TEST_F(TestQubicScoreTaskQueue, QueueIsReusableAfterReset)
+{
+ const unsigned int taskCount = 16;
+ for (unsigned int batch = 0; batch < 2; batch++)
+ {
+ queue().resetTaskQueue();
+ gTaskQueueProbe.reset();
+
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ const TaskQueuePayload task = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(countTaskRun, &task, sizeof(task)));
+ }
+
+ queue().runUntilDone(0);
+
+ for (unsigned int i = 0; i < taskCount; i++)
+ {
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "batch " << batch << " task " << i;
+ }
+ }
+}
+
+// Each task carries its own work function, so one batch can mix kinds. This is what lets a second
+// caller share the queue without changing it.
+TEST_F(TestQubicScoreTaskQueue, OneBatchCarriesDifferentWorkFunctions)
+{
+ // A second work function, so a batch can be shown to carry more than one kind of task.
+ const TaskQueueScoreFunction::WorkFunc countAltTaskRun = [](unsigned long long, void* payload)
+ {
+ const TaskQueuePayload* task = (const TaskQueuePayload*)payload;
+ task->probe->altRunCount.fetch_add(1);
+ };
+
+ const unsigned int pairCount = 32;
+ for (unsigned int i = 0; i < pairCount; i++)
+ {
+ const TaskQueuePayload counted = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(countTaskRun, &counted, sizeof(counted)));
+ const TaskQueuePayload alt = makeTaskQueuePayload(i);
+ EXPECT_TRUE(queue().addTask(countAltTaskRun, &alt, sizeof(alt)));
+ }
+
+ queue().runUntilDone(0);
+
+ for (unsigned int i = 0; i < pairCount; i++)
+ {
+ EXPECT_EQ(gTaskQueueProbe.runCount[i].load(), 1u) << "task " << i;
+ }
+ EXPECT_EQ(gTaskQueueProbe.altRunCount.load(), pairCount);
+}
diff --git a/test/test.vcxproj b/test/test.vcxproj
index fe0736f6f..b47d15232 100644
--- a/test/test.vcxproj
+++ b/test/test.vcxproj
@@ -176,9 +176,12 @@
+
+
+
diff --git a/test/trit_pack.cpp b/test/trit_pack.cpp
new file mode 100644
index 000000000..e8aef529e
--- /dev/null
+++ b/test/trit_pack.cpp
@@ -0,0 +1,137 @@
+#define NO_UEFI
+
+#include "gtest/gtest.h"
+
+#include "../src/mining/trit_pack.h"
+
+// Trit packing is a STORAGE format: the ant colony's ANN pool is packed in memory and written to
+// disk that way, so a layout change silently breaks snapshot loads rather than failing to compile.
+// The layout is therefore pinned by golden values here, not just round-tripped.
+//
+// trit_pack.h includes nothing, so this file does too - if that ever stops being true the test
+// stops building and the header has quietly grown a dependency.
+
+using score_engine::PackedTrits;
+
+// Two groups of five trits is small enough that every value the structure can hold fits in a loop,
+// so this is exhaustive rather than a sample: 3^10 assignments, each packed and unpacked.
+TEST(TestTritPack, RoundTripsEveryPossibleValue)
+{
+ using P = PackedTrits<2, 5>;
+ static constexpr unsigned int COMBINATIONS = 59049; // 3^10
+
+ for (unsigned int v = 0; v < COMBINATIONS; v++)
+ {
+ unsigned char src[P::tritCount];
+ unsigned int rest = v;
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ src[i] = (unsigned char)(rest % 3);
+ rest /= 3;
+ }
+
+ P packed;
+ packed.pack(src);
+
+ unsigned char back[P::tritCount];
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ back[i] = 0xFF; // so a trit the unpack never writes fails loudly
+ }
+ packed.unpack(back);
+
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ ASSERT_EQ(back[i], src[i]) << "value " << v << ", trit " << i;
+ }
+ }
+}
+
+// The documented layout - trit i of group g at bits [2i, 2i+2), lowest index in the lowest bits.
+// Round-trip tests pass under any self-consistent layout, so only fixed words catch a reordering
+// that would leave existing snapshot files unreadable.
+TEST(TestTritPack, LayoutIsTwoBitsPerTritLowestIndexFirst)
+{
+ PackedTrits<2, 4> packed;
+ const unsigned char src[8] = { 1, 2, 0, 1, 2, 2, 1, 0 };
+ packed.pack(src);
+
+ EXPECT_EQ(packed.word[0], 1ull + (2ull << 2) + (0ull << 4) + (1ull << 6)); // 73
+ EXPECT_EQ(packed.word[1], 2ull + (2ull << 2) + (1ull << 4) + (0ull << 6)); // 26
+}
+
+// A group is one scorer row and gets its own word, so editing a row must not touch any other. This
+// is what lets the colony reason about a neuron's LUT independently.
+TEST(TestTritPack, GroupsAreIndependent)
+{
+ using P = PackedTrits<4, 6>;
+ unsigned char src[P::tritCount];
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ src[i] = 0;
+ }
+
+ P base;
+ base.pack(src);
+
+ for (unsigned long long g = 0; g < P::groupCount; g++)
+ {
+ src[g * P::tritsPerGroup] = 2;
+ P edited;
+ edited.pack(src);
+ src[g * P::tritsPerGroup] = 0;
+
+ for (unsigned long long m = 0; m < P::groupCount; m++)
+ {
+ if (m == g)
+ {
+ ASSERT_NE(edited.word[m], base.word[m]) << "group " << m << " should have changed";
+ }
+ else
+ {
+ ASSERT_EQ(edited.word[m], base.word[m]) << "group " << m << " must not change";
+ }
+ }
+ }
+}
+
+// 32 trits is the widest group a uint64 holds, so the top trit sits at bits 62-63. A shift written
+// on a 32-bit type would be undefined there and would typically lose the high half silently.
+TEST(TestTritPack, WidestLegalGroupRoundTrips)
+{
+ using P = PackedTrits<1, 32>;
+ unsigned char src[P::tritCount];
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ src[i] = 2;
+ }
+
+ P packed;
+ packed.pack(src);
+ EXPECT_EQ(packed.word[0], 0xAAAAAAAAAAAAAAAAull); // every trit 0b10
+
+ unsigned char back[P::tritCount];
+ packed.unpack(back);
+ for (unsigned long long i = 0; i < P::tritCount; i++)
+ {
+ ASSERT_EQ(back[i], 2) << "trit " << i;
+ }
+}
+
+// pack() masks instead of validating. A byte the scorer should never have written is truncated to
+// its low two bits and stays inside its own trit - it does not shift the ones after it, which is
+// the property that keeps one bad byte from corrupting a whole row.
+TEST(TestTritPack, OutOfRangeByteCannotDisturbItsNeighbours)
+{
+ PackedTrits<1, 4> packed;
+ const unsigned char src[4] = { 4, 1, 7, 2 }; // 4 -> 0, 7 -> 3
+ packed.pack(src);
+
+ unsigned char back[4];
+ packed.unpack(back);
+
+ EXPECT_EQ(back[0], 0);
+ EXPECT_EQ(back[1], 1);
+ EXPECT_EQ(back[2], 3);
+ EXPECT_EQ(back[3], 2);
+}