Before reducing collected material to one row, I wanted the grouping decision to be inspectable. This small helper returns declared identity groups and title-review candidates separately.
Define the row contract before grouping
The input has exactly six fields. row_id names an observation within this call; source is web or mobile. namespace names a collection, owner is a synthetic publisher alias, record is a decimal record string, and title is a synthetic display title. The package contains no source parser, address reader or account connector. It is also not an anonymizer: arbitrary real data must not be passed through and assumed safe to publish.
A call accepts at most 64 rows and titles of up to 240 characters. Duplicate row IDs, unknown fields and missing fields reject the whole call. The caller must preserve rejected input separately. Assigning fresh row IDs merely to bypass a rejection would hide the evidence problem. Two real observations need distinguishable row aliases before this projection can preserve them.
Separate declared identities from unresolved observations
A declared identity is namespace plus lowercased owner plus the exact record string. Owner case folding applies only to the ASCII aliases accepted by this example. Namespace remains case sensitive, and leading zeros in record strings remain significant. The values 101 and 0101 therefore stay separate. This is an explicit sample policy, not a claim about the identifier rules of every provider.
If owner or record is missing, that observation receives its own unresolved group. A matching title does not attach it to another group. Multiple rows with the same complete declared identity share a group while every full input row remains in observations. Different titles across channels and repeated observations within one channel are retained rather than reduced to a preferred representative. Authentication of the supplied identity is outside this function.
Create title hints as a separate review projection
The title hint retains ASCII letters and digits plus Korean syllables after lowercasing. A-B and AB can collide, and an emoji-only title can become empty. This deliberately lossy transformation is visible in the implementation. Rows in separate groups can become a review candidate only when their namespace, owner and nonempty title hint match. Missing owners and empty hints produce no candidate.
No candidate does not establish that an item is new. Other writing systems or changed titles may fall outside this limited hint policy. Two complete identities sharing a hint produce title_collision; a bucket containing an unresolved group produces unresolved_title_candidate. Neither result selects a canonical record or authorizes a merge. A candidate list narrows a human review question without pretending to settle it.
Explore the implementation
Complete independent synthetic implementation
"""Independent synthetic grouping; no source parsing, authorization or writes."""
from copy import deepcopy
import re
FIELDS = {'row_id', 'source', 'namespace', 'owner', 'record', 'title'}
ALIAS = re.compile(r'[A-Za-z0-9_][A-Za-z0-9_.-]{0,47}\Z')
def title_key(title):
# Lossy and deliberately limited. This is a review hint, never an identity.
return re.sub(r'[^0-9a-z가-힣]+', '', title.lower())
def project(rows):
if not isinstance(rows, list) or len(rows) > 64:
raise ValueError('Expected a list of at most 64 synthetic rows')
clean, seen = [], set()
for original in rows:
if not isinstance(original, dict) or set(original) != FIELDS:
raise ValueError('Unexpected or missing fields')
row = deepcopy(original)
for field in ['row_id', 'namespace']:
if not isinstance(row[field], str) or not ALIAS.fullmatch(row[field]):
raise ValueError('Invalid alias')
if row['row_id'] in seen:
raise ValueError('Repeated row_id; preserve the evidence outside this call')
seen.add(row['row_id'])
if row['source'] not in ('web', 'mobile'):
raise ValueError('Unknown source')
if not isinstance(row['title'], str) or len(row['title']) > 240:
raise ValueError('Invalid title')
owner, record = row['owner'], row['record']
if owner not in (None, '') and (not isinstance(owner, str) or not ALIAS.fullmatch(owner)):
raise ValueError('Invalid owner alias')
if record not in (None, '') and (not isinstance(record, str) or not re.fullmatch(r'[0-9]{1,24}', record)):
raise ValueError('Record must be a decimal string or missing')
clean.append(row)
groups, buckets, row_group = {}, {}, {}
for row in sorted(clean, key=lambda x: x['row_id']):
known = bool(row['owner'] and row['record'])
identity = (row['namespace'], row['owner'].lower(), row['record']) if known else None
group_id = 'known:' + ':'.join(identity) if known else 'unresolved:' + row['row_id']
group = groups.setdefault(group_id, {
'group_id': group_id,
'basis': 'declared_identifier' if known else 'unresolved',
'declared_identity': list(identity) if known else None,
'member_row_ids': [], 'observations': []})
group['member_row_ids'].append(row['row_id'])
group['observations'].append(row)
row_group[row['row_id']] = group_id
key = title_key(row['title'])
if row['owner'] and key:
bucket = (row['namespace'], row['owner'].lower(), key)
buckets.setdefault(bucket, []).append(row['row_id'])
reviews = []
for (namespace, owner, key), members in sorted(buckets.items()):
ids = sorted({row_group[x] for x in members})
if len(ids) < 2:
continue
reviews.append({'namespace': namespace, 'owner_folded': owner,
'title_key': key, 'member_row_ids': sorted(members),
'group_ids': ids,
'reason': 'title_collision' if all(x.startswith('known:') for x in ids)
else 'unresolved_title_candidate'})
ordered = [groups[k] for k in sorted(groups)]
return {'groups': ordered, 'review_candidates': reviews,
'input_rows': len(clean),
'known_groups': sum(g['basis'] == 'declared_identifier' for g in ordered),
'unresolved_groups': sum(g['basis'] == 'unresolved' for g in ordered),
'identity_authenticated': False,
'normalized_title_proves_identity': False,
'merge_authorized': False, 'performs_external_write': False}
Read the output as an observation ledger
input_rows and the member_row_ids across all groups expose where each accepted observation went. observations contains exact deep copies of the six accepted fields. The output is detached from the input so a later display edit cannot silently change the caller's original rows. This preservation claim applies to accepted input; the function does not return a copy of rejected batches.
Rows, groups and candidate buckets are sorted to produce the same result across input order. Changing the supplied row IDs is a different operation and can change the result. There is no observation time in the contract, so this code cannot select the freshest title. A real adapter must preserve its own source ledger and define which material belongs in a namespace before relying on this projection.
Interpret tests and records within their synthetic scope
The package contains 27 test methods and five stored local execution cases. They cover changed titles under the same ID, matching titles under different IDs, a missing ID, an unresolved row beside two known anchors, and an empty lossy hint. One test compares all six permutations of three rows. Those permutations and subtest inputs are not added to the headline method count as extra tests.
The selector below reads records.json. It neither executes Python in the visitor's browser nor collects posts. After extraction, the same tests can be run with the standard library, and make_records.py regenerates the five outputs. This does not test the original collector, a live search service, a rendered interface or keyboard behavior. Those would require separate evidence from their respective systems.
Keep the adapter and authority decisions outside the helper
Extract the package and run python -B -m unittest -v. In Python, import project from identity_review and pass the synthetic row list. There is no file, server or browser connector. A production adapter must first verify source identifiers and decide how to represent its inputs under this contract. Title similarity must not directly trigger deletion, consolidation or a canonical-record update.
The fields identity_authenticated, normalized_title_proves_identity, merge_authorized and performs_external_write are always false. The existing source-specific rank observation module linked below addresses a later question: whether and where an item was observed. No automatic integration between the packages is implemented. The separation lets a reader reuse the identity-review idea without borrowing claims about source observation, authentication or operational authority.