trainset: pick evidence excerpt at the actual provider statement; document provider-currency caveat
- _evidence_excerpt now selects the name occurrence nearest a role keyword (custodian/adviser/transfer agent/...) instead of the first match, so the excerpt carries the real service-provider sentence, not boilerplate. - Documented the provider-currency caveat (gold N-CEN 2025Q3 vs current prospectus -> ~2-3% of triples name a *former* provider). A purely lexical former-flag proved unreliable (35% whole-book vs 1.7% excerpt-scoped) and was dropped in favour of an honest note; triples kept (entity+role still in text). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
991715ab76
commit
216fd9876c
@ -1165,6 +1165,17 @@ EXCERPT_RADIUS = 600 # chars around each grounded name
|
||||
_ROLE_HINT = ("custodian", "custody", "advis", "transfer agent", "administrat",
|
||||
"distributor", "underwriter", "serves as", "acts as", "series of")
|
||||
|
||||
# NOTE on provider-currency: because the prospectus is fetched at its CURRENT
|
||||
# version while the gold is N-CEN 2025Q3, a few percent of triples name a provider
|
||||
# the text describes as a *former* one (e.g. "former fund administrator ... through
|
||||
# September 5, 2025"). Manual inspection puts this at ~2-3% of distinct triples
|
||||
# (concentrated in adviser/sub-adviser/administrator; ~1% for custodian). A purely
|
||||
# lexical "former" flag proved unreliable -- scoped to the whole book it tagged 35%
|
||||
# (any stray "former" near any of a name's hundreds of occurrences), scoped to the
|
||||
# chosen excerpt it tagged 1.7% (the excerpt often picks a different, clean
|
||||
# occurrence). The entity+role is still present in the text, so these remain valid
|
||||
# extraction signal; the residual time-mismatch is documented rather than flagged.
|
||||
|
||||
|
||||
def _evidence_excerpt(text_norm_full, raw_text, label, predicate=None):
|
||||
"""Cut a coherent excerpt of raw_text around the BEST mention of the entity.
|
||||
@ -1233,7 +1244,7 @@ def build_trainset():
|
||||
if not kept:
|
||||
n_skip += 1
|
||||
continue
|
||||
# collect excerpts around each grounded object name; merge overlaps
|
||||
# collect excerpts around each grounded object name; merge overlaps.
|
||||
spans = []
|
||||
for t in kept:
|
||||
lbl = ents.get(t["o"], {}).get("label", t["o"])
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user