Methodology
How the numbers on Projam’s analysis pages are produced, from the platform’s records to each measure.
Describes bake format 4.1, search index 2, Rule Set V4.3. Wording awaits sign-off by the engine’s owner.
Every analysis comes from the SVC engine, which collects the Erasmus+ project records, curates and annotates their texts and computes each analysis once per update. Projam shows those results unchanged. Two things are computed when you ask: the trend count, proven equal to the engine’s own by golden tests, and the free-search concordance, which follows the engine’s sampling rule and is checked by an oracle run.
Principles
- Complete population, not a sample: every project the platform publishes is collected and checked against its own counts.
- Mark, do not delete: removed texts stay in the corpus with the reason recorded.
- Rules, not inspection: every decision is a versioned rule or word list applied to all projects.
- Reproducible by version: each build records the collection, curation and rules it comes from.
- Count by project: a project counts once, however often it repeats a phrase.
- Calculate on the annotated layer, show the original text.
Corpus
Collection
Source: the European Commission’s Erasmus+ Project Results Platform, call years 2014–2026, both programme periods. Per project two records are kept: the search record (classification, application texts) and the project card (report summary, participants). A collection counts as complete only when the total, the per-year counts, the absence of duplicates and the presence of every card are confirmed. Organisation data keeps name, PIC, country, role and type; contact details are never collected.
Curation
Ten narrative fields are read in a fixed order, application texts before report texts. Each is normalised and split into paragraphs and sentences. A field is removed when it is a placeholder, a platform text (identical in at least 100 projects of at least 50 coordinating organisations), a copy of another project’s field or a duplicate within the project. Language is identified with CLD3; a field counts as English at 70% English words or more.
A project is a text project when its kept English fields hold at least 200 words (the rule of bake 4.1); otherwise it is metadata-only: its record is kept, and it takes no part in text analyses. The counts of the build shown are on the Corpus page.
Annotation
Every analysed word carries its form, lemma, part of speech, content mark and position. spaCy 3.8.16 (en_core_web_sm 3.8.0) tags and lemmatises; rule corrections fix its known errors on this corpus. Hyphenated words are one word; hyphen families are joined into the most written spelling; the spaced spelling joins only when every part is a content word on its own. Spelling is mapped to American English (VarCon). Content words exclude function words and a reviewed list of administrative words.
Units
- Phrase: one to three adjacent content words within a sentence, never across punctuation, shown in its most frequent written form.
- Project frequency: the text projects whose analysed text contains the phrase.
- Group: an action type, a family of action types that joins a 2014–2020 type with its 2021–2027 successors (a reviewed mapping), or the whole corpus. Group size is its text projects.
Phrases
Notation for a phrase and a group: a = group projects using it, A = group text projects, b = projects outside using it, B = text projects outside, N = A + B.
Distinctive phrases
A phrase is listed when its log-likelihood G² is at least 15.13 (p < 0.0001), a ≥ 5 and the Log Ratio is above 0. G² = 2 Σ O·ln(O/E) over the 2×2 table of projects with and without the phrase, inside and outside the group, with E = row total × column total / N. The list is ordered by the SimpleMaths score:
score = (100·a/A + k) / (100·b/B + k), k = 1
The added k keeps phrases used by a handful of projects from ranking high on small numbers alone. The score is a ranking key, not “times more frequent”. Each row also shows the share in the group and outside it, G² and the Log Ratio.
Log Ratio = log2 [(a/A) / (b/B)], a zero count taken as 0.5
A Log Ratio of 1 means twice as widespread in the group; 3 means eight times. G² measures the strength of evidence, not the size of the difference.
Common phrases
The phrases with the largest share a/A (with a ≥ 5): what a group discusses most, whether or not other groups do too. The whole corpus has common phrases only; compared with itself it has no distinctive ones.
Which phrases are listed, and hidden rows
Phrases in fewer than 10 projects of the corpus are not considered; lists hold up to 200 rows per group, phrase length and measure. Some rows are not shown, because the list merges them with another row or they are part of one:
- phrases that, in at least 90% of the projects using them, occur as part of a listed phrase one word longer;
- pieces of names (a three-word phrase with the same word on the same side of it in at least 90% of its projects; an article, a possessive or a preposition beside it does not count);
- phrases written the same way as a higher-ranked row.
Their sentences can still be opened in the concordance, and their trend can still be searched.
Concordance
The concordance (Bağlam) shows sentences with the phrase marked. It always says which of three tiers answered.
- Listed phrase. The phrase is in the group’s lists: the engine’s own view, with its statistics (projects, sentences, share, score, G², Log Ratio). A phrase hidden from the list (see above) is also answered here, with a note.
- Free search. The phrase is not in the lists: the curated corpus is searched directly with the same reading of the query as the trend. The page states that the phrase is not among the prepared lists and that its distinctiveness was not measured. The project count equals the trend’s total for the same query and group; sentence counts are not given.
- No match. The page names the reading that was searched, the group and the build, so a reading that differs from your intention can be recognised. It never reports a phrase as absent merely because the lists do not hold it.
Sampling
Lines are a fixed sample, not ranked by relevance: at most one per project, projects in the order of a hash of the phrase and the project, 25 lines per view. The free tier follows the same rule and shows at most 25 lines.
Content words and any words
Lists and views follow the engine: a phrase is made of content words, so a listed view matches content words only. In the free search, the words you type are all matched, stop words included, because this is what the trend counts (youth at risk counts the word at); so the free search finds some sentences a listed view for the same words would not. Both read the query the same way and match only consecutive lemmas within one sentence.
Check against the engine: the listed views count content-word runs; the free tier matches any word, which agrees with the trend count. An oracle run of the free-search procedure in content-word mode on the listed phrases of ten groups must reproduce the engine’s views exactly; it is a release gate, and its results are recorded in the roadmap’s gate table.
Project pages
A project page shows the published text of a text project field by field, in the curated order, with removed fields and their reason, headings marked and the concordance sentence highlighted. Source: Erasmus+ Project Results Platform, European Commission.
Collocation
For each one-word entry of a group’s lists (the node), the collocates are content words within ±5 words in the same sentence. Association is measured with logDice (Rychlý 2008):
logDice = 14 + log2 [2·fxy / (fx + fy)]
fx and fy are the occurrences of node and collocate in the group; fxy is their co-occurrence: per sentence, the smaller of nx (the node’s words that have the collocate within ±5 words) and ny (the collocate’s words that have the node within ±5 words), summed over the group’s sentences. The maximum is 14; a difference of one point is a doubling of the co-occurrence relative to the two frequencies. logDice does not depend on group size, so groups can be compared, and it does not favour rare words as Mutual Information does.
A collocate must co-occur with the node in at least 5 projects; 20 collocates are listed per node. Each row shows the share of the collocate’s positions left of the node. Each pair has its own concordance of 25 lines, with both words marked.
Trend
For a term you type, per call year: the number of text projects whose text contains the term, divided by the year’s text projects within the same filters.
py = ay / Ny × 1,000
ay counts a project once if the term occurs anywhere in its kept text, application or report, as consecutive words within one stretch of text. A 95% Wilson score interval (Wilson 1927, z = 1.96) is drawn around the rate; it stays within 0–1 and behaves with few projects and zero counts. A year without projects has no value, not zero. Filters are the group, field of education, topic, the coordinator’s organisation type and country; they narrow both counts. The 2021 programme boundary is marked on the chart; the line is not broken.
How the query is read
The term goes through the engine’s own lemmatiser with the build’s Rule Set: inflection, hyphen families, American spelling and one reading per -ing/-ed pair. Organisations is searched as organization, wellbeing as well-being. Only the same lemmas count; related words are separate queries; stop words in the query count; a match never crosses a sentence end or punctuation. Projam shows the reading under the search box.
Builds and versions
The engine runs again at a set interval (quarterly by default) and publishes a new build after automatic checks and approval by a platform administrator; the previous build stays available for rollback. Every analysis number names its build. In the planned team workspace, saved references will store the phrase as written, the project reference and the build, and will show old and new values side by side after an update.
A page keeps showing the build it was loaded with. When a newer build is published, the page says so and offers to switch; links carry the build so that a shared link shows what its author saw.
| Bake format | Search index | Rule Set | Notes |
|---|---|---|---|
| 4.1 | 2 | V4.3 | SimpleMaths ranking; phrase breaks at punctuation; per-sentence logDice; duplicates and name pieces hidden; trend with Wilson intervals; text projects need 200 words |
A newer engine version is shown only after this page and the readers are updated.
Known limitations
| Limitation | Effect |
|---|---|
| Language identification on short or mixed text | Short or bilingual fields can be misjudged; fields below 70% English are removed whole. |
| Tagging errors | The small spaCy model mistags some words, which affects their content mark. |
| Changing action types | Families join the two programme periods through a reviewed, not an official, mapping. |
| Text availability by year | Some actions carry only the platform’s standard text in some years; recent years have few final reports, so the trend’s base is smaller and its interval wider. |
| Form changes | Shorter forms give a project fewer chances to mention a term; not corrected. |
| Trend and free-search queries | Only the query’s own lemmas match; senses are not separated. |
| Free search | Not a measure of distinctiveness; lines are a fixed sample, not ranked by relevance. |
| Platform as published | Texts the platform does not publish are not seen. |
References
Dunning (1993); Rayson & Garside (2000); Hardie (2014); Kilgarriff (2009); Rychlý (2008); Wilson (1927); Gries (2008); Honnibal et al. (2020); Google CLD3; Atkinson & Titze (VarCon); Porter (Snowball stop list). Full entries and every rule with examples are in the engine’s own documents, which are authoritative.