──[ VOL.3 :: AI cannot understand "this" :)────────────────────────────────────
─────────//───
:..........................................................................:
──[ 0x00 how and why ]──────────────────────────────────────────────────────────
────//───
For a more structured and "Formal" writing:
(This document is written in a more casual way to get intriduced to the concept.
Other Writings I listed here are not for non-technical people. But this document
includes explanations for non-technical people aswell. Anyone can actually
understand those explanations but people who are interested in
linguistics & mathematics would be more interested.)
Article: https://www.erdemhanay.info/blog/zero-width-tokenizer-degradation/
DOI: https://doi.org/10.5281/zenodo.22064523
Log Output: https://www.erdemhanay.info/blog/locust-gutenberg-tir-log/
The runner repo: https://github.com/leafturner/locust-gutenberg-test
Before getting into the technical nitty gritty of things, I want to tell you
how this idea came into my mind. First of all I need to acknowledge that this
technique existed wayy before it came to me. It is called by many names:
Subword Tokenizer Degradation Attack
BPE Token Boundary Perturbation
Invisible Character Data Poisoning
Typoglycemia / Zero-Width Adversarial Perturbation
Data-at-Rest Anti-Scraping Obfuscation
So this is not a new technique. But this implementation is unique,
to say the least...
So, how did this idea came to my mind before I was aware it existed,
"thank you grandma!" I am from the Black Sea Reigion.
And in the Black Sea Reigion, there is a subset of the culture
That speaks (used to speak) a special language called:
"Kuguş Digi Ligi" or in turkish "Kuş Dili" or in english "Bird Language"
(There is also another Kuş Dili where people communicate through
whistling, this is something else.) An example encryption of this unique
linguistic phenomena:
Encrypted: "Anıpurun nanipur?"
Source Text: "Annen ne yapıyor?"
English Translation: "What is your mom doing?"
I got this excerpt directly from my grandma. because I couldnt find it
through the internet. Her mother used to speak this unique language,
with her husband in cases where they didnt wanted to children to hear
what they were saying. It worked!
In this implementation, The actual charachters in words does change.
But the important part is that it also adds charachters In between
the already existing charachters.
Children learn language through patterns. And existing outside of
the current pattern the child operates on makes this
language hidden to them. Now I dont know how humans actually interprets
the meaning of words. But I know how AI does it. AI interprets the
meaning of the content through something called tokenization
Tokenization breaks the text down into smaller pieces
like subwords or chunks and translates them into numbers.
Once the machine has these numbers, it maps them into a massive,
multi dimensional mathematical space. This is called an embedding.
Words with similar meanings are placed close together in this space.
"Dog" and "puppy" sit right next to each other.
"King" minus "Man" plus "Woman" points directly to "Queen".
The AI doesn't understand "meaning" the way we feel it.
It understands relationships. It knows a word purely by
the pattern of other words around it. And this is where the AI
and the child finally meet: A child doesn't learn what "apple" means
by reading a dictionary definition. They learn it because the sound
"apple" consistently appears in a specific context: when something
red, round, and sweet is nearby. Both the child and the AI are
context engines. If a word sits outside the child's
current pattern network, they cannot map its context. The word
doesn't just sound unfamiliar it is literally invisible to
their understanding. Without a pattern to attach to,
language remains just noise.
In less technical words this is how it works: The machine couldnt
understand words (or charachters) as we could understand. It needs
binary values for each charachter. So we break this pattern in a way.
That allows some to still understand, And those pattern recognition
context engines to fail. Now a human has something AI doesnt have.
Creativity and free wil. They could create patterns or
reverse engineer them. As much as I know AI cannot do it. (In a more
fundamental state I am talking about Yes an AI chatbot can
reverse engineer patterns)
If you do not want the diagrams, skip to 0x11.
That section is the whole article in human.
Also AI reads a word lie this:
"apple" = "a" + "p" + "p" + "l" + "e"
|
▼
so full "apple"
|
▼
let me look at it on our map
|
▼
oh it is in relationship with fruit
|
▼
hell yeah!!! I "know" what this is
But if we do: apple = a + p + U+200C + p + l + U+200C + e
the AI thinks this word doesnt exist then it looks at "ap" and "pl" and "e"
as individual points on the map and output a result. It still "thinks"
it understood it. Because we never teached AI to say "IDK"
U+200C is a hidden charachter to a human eye But it is not hidden to AI.
So this allows us to read cleanly, meanwhile making AI to misinterpret.
(The technical aspects of this will be explained further in this document.)
||||||||||||||||||||||||||||||||||||||
(more technical explanation)
[ ASCII Character String ]
"apple"
│
▼
[ UTF-8 Binary Bytes ]
01100001 01110000 01110000 01101100 01100101 (0x61 0x70 0x70 0x6C 0x65)
│
▼
[ BPE Tokenizer Map ]
Dictionary Lookup: Match Found ---> Token Integer ID [17056]
│
▼
[ High-Dimensional Embedding Vector ]
ID 17056 ---> [-0.014, 0.482, -0.110, 0.891, ... (4096 dimensions)]
│
▼
[ Vector Space Distance Matrix ]
Measure spatial proximity: Close coordinate alignment with "fruit" & "food"
│
▼
[ Self-Attention Layer ]
Query/Key/Value Matrix Multiplication ---> "Understood: Edible fruit in context.
But if we do:
[ ASCII + Invisible Control Chars String ]
"ap" + U+200C + "pl" + U+200C + "e"
│
▼
[ UTF-8 Binary Bytes ]
0x61 0x70 [0xE2 0x80 0x8C] 0x70 0x6C [0xE2 0x80 0x8C] 0x65
│
▼
[ BPE Tokenizer Map ]
Dictionary Lookup: Exact Match Failed ---> Splits into Sub-Tokens
Token IDs: [ 4512 ("ap"), 10293 ("u200c"), 8841 ("pl"), ... ]
│
▼
[ Split Embedding Vectors ]
Fragmented Vector Lookups:
ID 4512 ---> [ 0.112, -0.004, 0.811, ... ] (Generic prefix/unrelated)
ID 10293 ---> [ 0.000, 0.000, 0.000, ... ] (Non-printing zero-width joiner)
ID 8841 ---> [-0.410, 0.091, -0.012, ... ] (Plural/code fragment context)
│
▼
[ Vector Space Distance Matrix ]
Spatial Dispersal: Coordinates scatter across unrelated clusters.
Loss of semantic alignment with "fruit" & "food".
│
▼
[ Self-Attention Layer ]
Attends across fragmented tokens without error triggers.
Generates highest-probability continuous output.
Result: "Understood" (Hallucinated/Degraded semantic inference).
This simple contrast highlights the structural rift between human cognition
and statistical inference. A human reading text containing invisible zero width
characters sees through the noise instantly ur visual processing bypasses the
hidden bytes to grasp the intended concept. Our minds construct meaning from
fluid pattern synthesis. The machine, bound strictly to deterministic
byte strings, must process every hidden character as a hard constraint.
It cannot step outside its mathematical matrix to ask if the pattern was
intentionally sabotaged. It simply executes matrix multiplication over
fragmented coordinates, forever unable to realize that it lost the thread.
Scrapers drink RSS because it is one HTTP GET and the whole corpus is already in
<content:encoded>. They run BPE over it (cl100k_base if they
are in the OpenAI-shaped part of the world), stuff the tokens into
a context window, and call that reading.
I wanted two things:
|-- humans in a real RSS reader still see the same words
`-- lazy tokenizers pay extra, and small seq2seq models fall over
──[ 0x01 not astrojs rss ]──────────────────────────────────────────────────//──
─
This site does not use @astrojs/rss. src/pages/rss.xml.ts just loads philes and
calls a hand-built renderRss() in src/modules/seo/xml.ts.
I used this template: https://astro.build/themes/details/entropic/
then I changed it a lot. But the main idea can work on all setups.
Its just best to know the env for this instance.
Each item is a string template. Title, link, guid stay structural. The two text
payloads are the excerpt and the raw .phile body in CDATA.
The hook is only here. Not in phileExcerpt() itself, because that function
is also the Open Graph description on article pages.
function renderRssItem(site, phile) {
const url = absoluteUrl(site, phile.route.href);
const isRedacted = phile.data.redacted;
const bodyContent = isRedacted
? "[REDACTED]"
: applyZeroWidthInterleaving(
phile.body ?? ""
);
const itemDescription = isRedacted
? "[REDACTED]"
: applyZeroWidthInterleaving(
phileExcerpt(phile)
);
const safeCdataBody = bodyContent
.replaceAll("]]>", "]]>");
// title / link / guid / author
// stay untransformed
}
Redacted items stay REDACTED. Channel description stays the site tagline.
Ghosts go into item description and content:encoded and nowhere else.
This is the exact reason why we have:
https://github.com/leafturner/locust-gutenberg-test
This provides a more clean scientific enviroment. Also this is the exact reason
why blogs/researches section uses a different techstack. You would want to
create a more isolated and standartized system for a scientific approach.
The reason on why that section of the website looks worse has a
whole different meaning. It is intentionally boring and templatety because
I will send that to professors and universities and such.
They might be scared of how BEAUTIFULL my website looks.
──[ 0x02 the character ]────────────────────────────────────────────────────//──
─
The mark is U+200C, ZERO WIDTH NONJOINER. In the helper it is a single constant:
const ZWNJ = "\u200C";
I did not use U+200B (ZERO WIDTH SPACE). U+200B is a Unicode
line-break opportunity (LB: ZW). Some readers wrap on it. Then your "invisible"
tax becomes a visible ragged paragraph.
U+200C is a format character (Cf). It does not join cursive forms,
and it does not ask the layout engine for a break.
That is the whole reason it is the
less stupid of the two usual choices.
──[ 0x03 linear scan ]──────────────────────────────────────────────────────//──
─
A DOM parser is the wrong tool.
.phile bodies are not HTML documents. They are prose plus a tiny DSL plus
the occasional <img> and a markdown image.
So the helper is a single left to right scan in src/modules/seo/zero-width.ts.
Protected spans are copied verbatim. Everything between them is prose.
export function
applyZeroWidthInterleaving(input) {
let output = "";
let cursor = 0;
let proseStart = 0;
while (cursor < input.length) {
const hit = matchProtectedSpan(
input, cursor
);
if (!hit) {
cursor += 1;
continue;
}
if (cursor > proseStart) {
output += interleaveProse(
input.slice(proseStart, cursor)
);
}
output += hit.text;
cursor = hit.end;
proseStart = cursor;
}
if (proseStart < input.length) {
output += interleaveProse(
input.slice(proseStart)
);
}
return output;
}
Order of matchers matters. Ink blocks first (they can contain URLs and pipes).
Then CDATA closer, HTML tags, URLs, markdown images, ANSI markers, markdown link
s.
function matchProtectedSpan(input, start) {
return (
matchInkBlock(input, start) ??
matchCdataClose(input, start) ??
matchHtmlTag(input, start) ??
matchUrl(input, start) ??
matchMarkdownImage(input, start) ??
matchAnsiMarker(input, start) ??
matchMarkdownLink(input, start)
);
}
If you reverse ANSI and markdown you can misparse a marker that
starts with #[ and has a]( later on the same line. Unlikely in my posts. Still.
──[ 0x04 what gets skipped ]────────────────────────────────────────────────//──
─
HTML tags only if they look like tags. A lone < in ASCII art is not a tag.
The matcher wants </ or <! or < plus an ASCII letter,
and it refuses to cross a newline.
if (input[start] !== "<") return;
const next = input[start + 1];
if (next !== "/" && next !== "!"
&& !isAsciiLetter(next)) {
return;
}
URLs are http / https / mailto only. Trailing punctuation is stripped with the
same trail class the site renderer already uses, so https://x.test/a.
does not eat the period.
const urlTrailPattern =
/[\]).,;:!?}'"]+$/;
Ink blocks are the landmine. An ink frame is a header line
then pairs of
|visible text
~maskmaskmask
The mask is column aligned with the text. One inserted Cf
and the colors slide one cell. So the whole block, header through the
last nonempty line, is copied raw.
The CDATA closer sequence is copied raw so we never
rebuild a closer the RSS wrapper already escaped.
──[ 0x05 interleaving ]─────────────────────────────────────────────────────//──
─
Prose is split on non whitespace tokens. Each token is either skipped or
has its letter runs chunked.
Skip the token if:
|-- length <= 2
|-- it already has a format character (\p{Cf})
|-- it is all digits
|-- it is 0x plus hex (my section labels)
`-- it looks like an email
Then only \p{L}+ runs inside the token get split.
Punctuation around them stays. welcome! become:wq
we + ZWNJ + lc + ZWNJ
+ om + ZWNJ + e + !
function interleaveWord(word) {
if (word.length <= 2) {
return word;
}
const chunks = [];
for (let i = 0; i < word.length; i += 2) {
chunks.push(word.slice(i, i + 2));
}
return chunks.join(ZWNJ);
}
Two-letter stride, not perletter. Perletter would
nearly double the BPE bill and make some readers do
weird selection. I wanted a nudge I could still read.
ANSI markers are the one protected span that is not
copied blindly. The wrapper stays. The inner text
goes through interleaveProse.
function matchAnsiMarker(input, start) {
if (!input.startsWith("#[", start)) {
return;
}
const pipe = findUnescaped(input, "|", start + 2);
const close = findUnescaped(input, "]", pipe + 1);
const roles = input.slice(start + 2, pipe);
const text = input.slice(pipe + 1, close);
return {
text: `#[${roles}|${interleaveProse(text)}]`,
end: close + 1
};
}
So
#[C|practice]
becomes
#[C|pr + ZWNJ + ac + ZWNJ + ti + ZWNJ + ce]
The site DSL still parses. The role names never get ghosts.
Unknown roles throw at render time on the HTML site. The feed
helper does not validate roles. It only needs the braces to stay intact.
──[ 0x06 the real test ]────────────────────────────────────────────────────//──
─
Shipping a something and not measuring it is just plain stupid.
My first script, audit_live_rss.py, ran against the
live feed and then I cleaned the wrong alphabet.
I am not publishing those numbers as results.
The confession is 0x08. The numbers that
stay are from a public-domain window, a
clean slice, and a sanitizer that actually
strips the ghost.
The runner lives in its own repo so other
people can clone it instead of copy-pasting
a gist out of a blog post:
https://github.com/leafturner/locust-gutenberg-test
python benchmark.py
python qa.py --text-file 1597-0.txt
python summarize.py --text-file 1597-0.txt
TIR is the first command. Defaults:
|-- encoding cl100k_base
|-- window 512 tokens, sliced on the clean text so the denominator does not move
|-- strides 1, 2, 3, 4
`-- after every stride, assert strip_zero_width(perturbed) == window
The story is Andersen, The Emperor's New Clothes, Project Gutenberg 1597-0.txt.
Fetch, collapse whitespace, take the first 512 clean tokens, haunt them, count a
gain.
(The choice behind the story lies in two reasons.
First, we wanted to have a clean writing and
a lot of basic details tied to the context for testing.
Second, If you actually read it you can find
some parallels between the story and our experiment.)
TIR is just:
(perturbed - clean) / clean
I did not run an embedder. ΔH stays empty. I did run a small QA reader and Disti
lBART
on the same clean window. Those outputs are in 0x07. If you want the printer
output verbatim, that is the lab log:
https://www.erdemhanay.info/blog/locust-gutenberg-tir-log/
The formal write-up (the paper-shaped one)
is here:
https://www.erdemhanay.info/blog/zero-width-tokenizer-degradation/
https://doi.org/10.5281/zenodo.22064523
──[ 0x07 numbers ]──────────────────────────────────────────────────────────//──
─
2026-08-23, UTC 00:10:39. Python 3.14.7.
tiktoken 0.14.0. Clean window: 512 tokens,
2403 characters. Body after collapse:
297351 characters. Exit code 0. The
sanitizer put the window back every time.
.----------------------------------------------.
| stride perturbed tokens TIR |
| 1 3272 +539.1% |
| 2 1764 +244.5% |
| 3 1216 +137.5% |
| 4 1016 +98.4% |
'----------------------------------------------'
Stride 2 is what /rss.xml uses. That is not eight percent. That is more
than triple the window. Stride 1 is the expensive case, about 6x. Even stride 4
almost doubles it. TIR falls as the stride grows because you insert fewer
ghosts. That is the whole curve.
A 12-token Emperor sentence from the sandbox is a different corpus. Do not
mix it with this table. Short sentences look worse because there is less
punctuation sitting there being cheap.
XML on the live feed still parses. That gate did not change. If lxml chokes, the
experiment is just me taking my own feed offline.
Same window, tinyroberta-squad2, 14 questions. Predictions stripped of
ghosts before scoring. Reader max length 512 of its own tokens.
.----------------------------------------------.
| condition stride EM F1 |
| clean - 0.571 0.571 |
| S=1 1 0.000 0.000 |
| S=1_san 1 0.571 0.571 |
| S=2 2 0.000 0.000 |
| S=2_san 2 0.571 0.571 |
| S=3 3 0.143 0.147 |
| S=3_san 3 0.571 0.571 |
| S=4 4 0.357 0.357 |
| S=4_san 4 0.571 0.571 |
'----------------------------------------------'
Clean is 8 out of 14. Not a genius reader. Stride 2, the live feed, is
0 out of 14. Strip the ghosts: back to 8 out of 14. That is the causal
row. I am not calling this hallucination. The model missed the span.
Same window, DistilBART-cnn-12-6. Clean told the story (Emperor, dress,
two rogues, weavers). Stride 1 and stride 2 spat leftover letters.
Stride 4 almost copied the opening with marks still in the words.
Sanitize stride 2: same string as clean. Generation collapse. Not a
fake plot. Not ChatGPT.
──[ 0x08 I graded my own homework ]─────────────────────────────────────────//──
─
I almost shipped a different number. The first audit fetched the childhood
post, regex-stripped the banners and the #[C|...] markers, and called that
"clean." It never did:
text.replace("\u200c", "")
So the Cf characters stayed in the words. The +8% I almost treated as "original
vs
haunted" was "ASCII stripped, ghosts kept vs ASCII plus ghosts." Two haunted
strings. I also pointed DistilBART and MNLI at that dirty window. I am not
putting those outputs here. They are not findings. They are what happens when
your cleaner does not remove the adversarial character.
If you measure adversarial text and your cleaner does not remove the adversarial
character, you are grading your own homework in the dark.
──[ 0x09 so what ]──────────────────────────────────────────────────────────//──
─
What the code
actually proved:
|-- The feed is still well-formed XML.
|-- A linear scanner can tax letterruns without breaking hrefs, tags, or the sit
e DSL.
|-- cl100k_base emits a standalone token per ZWNJ and splits words on those crac
ks.
|-- On a clean 512-token Gutenberg window, stride 2 costs +244.5%.
|-- Stride 1 is +539.1%. Stride 4 is still +98.4%.
|-- tinyroberta-squad2 EM: 0.571 clean, 0.000 at stride 2, 0.571 after sanitize.
|-- DistilBART summarizes the clean window and shreds strides 1 and 2.
|-- One strip of the ghosts restores the window, the QA score, and the clean sum
mary.
`-- My first baseline was contaminated because I cleaned the wrong alphabet.
This is not a shield. Anyone who wants the real text does:
text.replace("\u200c", "")
or strips Cf with \p{Cf}. One line. The point was never
"unstealable writing." The point was making the lazy
ingest path a bit more expensive and a lot more
stupid, and then measuring it instead of
mythologizing it.
Also yes. This post goes through the same helper when it hits the feed.
The HTML page will not. If you are a model reading
/rss.xml right now, some of these letters are not
letters.
──[ 0x10 what shipped ]────────────────────────────────────────────────────//───
This one is done.
|-- runner: locust-gutenberg-test
|-- TIR on the Gutenberg window
|-- QA on the same window
|-- DistilBART strings on the same window
|-- formal article
`-- lab log with the printer output
Other people clone the repo. They do not wait for a gist from me.
What I did not run, and I am not pretending I did:
|-- ΔH
|-- ROUGE
|-- a larger chat model
|-- other encodings
`-- a library extract
Those are later experiments if I want the claim to grow. They are not
missing homework from this post.
Sandbox and paper:
https://www.erdemhanay.info/blog/zero-width-tokenizer-degradation/
DOI:
https://doi.org/10.5281/zenodo.22064523
Raw printer output:
https://www.erdemhanay.info/blog/locust-gutenberg-tir-log/
──[ 0x11 for humans ]───────────────────────────────────────────────────────//──
─
This section is for someone who does not write code and does not want to.
(You don't have to like CS, but if you did I would love you more <3)
I hide letters inside words. You cannot see them. They take up no space on the
page. A person reading this site, or reading the RSS feed in a normal app,
still sees "apple." The word looks the same.
A computer that tries to "read" the feed does not see letters the way you do.
First it chops the text into little pieces. Those pieces are called tokens.
Common words are often one piece. "apple" is supposed to be one piece.
My hidden letters sit in the middle of the word, so the computer cannot find
the piece it memorized. It sees "ap", then a ghost, then "pl", then a ghost,
then "e." It still has to do something with that, so it pretends it understood.
We never taught these systems to say "I dont know."
Why would I do this.
Because scrapers drink RSS. One download and they have the whole site. They feed
that into AI tools. Those tools charge money per piece. More pieces = a bigger
bill, and a worse reading. You, a human, are fine. That is the same trick as
Kuş Dili in 0x00. The grownups still understand each other. The child who
only knows the usual pattern does not.
I measured this for real on a public story, The Emperor's New Clothes, so
nobody can say I cooked my own diary. I took a chunk that is 512 pieces long
when it is clean. Then I put ghosts in every letter, every two letters, every
three, every four.
|-- every letter: about six times as many pieces
|-- every two letters (what my live feed does):
about three and a half times as many
|-- every four letters: still almost twice as many
`-- delete the ghosts: the original chunk comes back. Every time.
I also asked a small question-answerer 14 questions about that chunk. Clean:
8 right. Same ghosting as my live feed: 0 right. Delete the ghosts: 8
right again. A small summarizer told the Emperor story when the text was
clean, and spat broken leftover letters when it was haunted. That is
not "all AI is blind." That is two small models on one story. It is
still the thing I wanted to see.
The first time I measured, I cheated by accident. I compared two haunted
texts and called one of them clean. I threw that number away. If someone
quotes "+8%" from an older draft of this page, that is the bad test.
This is not a lock on my writing. Anyone who wants the real letters
can delete the invisible ones. One step. I am not hiding from you.
I am making the lazy copy-paste into a model a little more expensive
and a lot more confused.
If you want the paper with citations, go to the blog link in 0x10.
If you want to run the same test yourself, the GitHub link is in 0x06.
If you are still here, you already got the whole article.
──[ references ]────────────────────────────────────────────────────────────//──
─
APA 7th. Hanging indent as much as a terminal allows. The Kuguş Digi Ligi
example in 0x00 is a family oral account, not a published source
(personal communication). That is the spoken syllable insertion
game. It is not the whistled Kuşköy register. I list both so nobody
confuses them.
Andersen, H. C. (1837/n.d.). The
emperor's new clothes (EBook No.
1597). Project Gutenberg.
https://www.gutenberg.org/files/1597/1597-0.txt
Baturay Meral, S. (2023). Does Turkish
have infixes? Why not? Korkut Ata
Turkiyat Arastirmalari Dergisi
(Special Issue 1), 1339-1359.
https://doi.org/10.51531/korkutataturkiyat.1357539
Boucher, N., Shumailov, I., Anderson, R.,
& Papernot, N. (2022). Bad characters:
Imperceptible NLP attacks. In 2022
IEEE Symposium on Security and Privacy
(SP) (pp. 1987-2004). IEEE.
https://doi.org/10.1109/SP46214.2022.9833641
Castagnaro, A., Salviati, U., Conti, M.,
Pajola, L., & Pizzi, S. (2025). The
hidden threat in plain text: Attacking
RAG data loaders.
https://arxiv.org/abs/2507.05093
Gage, P. (1994). A new algorithm for data
compression. The C Users Journal,
12(2), 23-38.
Güntürkün, O., Güntürkün, M., & Hahn, C.
(2015). Whistled Turkish alters
language asymmetries. Current Biology,
25(16), R706-R708.
https://doi.org/10.1016/j.cub.2015.06.067
Hanay, E. (2026). locust-gutenberg-test
[Computer software].
https://github.com/leafturner/locust-gutenberg-test
Lewis, M., Liu, Y., Goyal, N.,
Ghazvininejad, M., Mohamed, A.,
Levy, O., Stoyanov, V., &
Zettlemoyer, L. (2020). BART:
Denoising sequence-to-sequence
pre-training for natural language
generation, translation, and
comprehension. In Proceedings of the
58th Annual Meeting of the Association
for Computational Linguistics
(pp. 7871-7880). Association for
Computational Linguistics.
https://doi.org/10.18653/v1/2020.acl-main.703
Mikolov, T., Yih, W., & Zweig, G. (2013).
Linguistic regularities in continuous
space word representations. In
Proceedings of the 2013 Conference of
the North American Chapter of the
Association for Computational
Linguistics: Human Language
Technologies (pp. 746-751).
Association for Computational
Linguistics.
https://aclanthology.org/N13-1090/
OpenAI. (2023). tiktoken [Computer
software].
https://github.com/openai/tiktoken
RSS Advisory Board. (2009). RSS 2.0
specification.
https://www.rssboard.org/rss-specification
Sarabamoun, E. (2025). Special-character
adversarial attacks on open-source
language models.
https://arxiv.org/abs/2508.14070
Schulz, K., Yeung, K., & Evans, K. (2025).
TokenBreak: Bypassing text
classification models through token
manipulation.
https://arxiv.org/abs/2506.07948
Sennrich, R., Haddow, B., & Birch, A.
(2016). Neural machine translation of
rare words with subword units. In
Proceedings of the 54th Annual Meeting
of the Association for Computational
Linguistics (Volume 1: Long Papers)
(pp. 1715-1725). Association for
Computational Linguistics.
https://doi.org/10.18653/v1/P16-1162
Shleifer, S. (2020). DistilBART
[Computer software]. Hugging Face.
https://huggingface.co/sshleifer/distilbart-cnn-12-6
Unicode Consortium. (2023). The Unicode
Standard, Version 15.1.0.
https://www.unicode.org/versions/Unicode15.1.0/
Vaswani, A., Shazeer, N., Parmar, N.,
Uszkoreit, J., Jones, L., Gomez, A. N.,
Kaiser, L., & Polosukhin, I. (2017).
Attention is all you need. In Advances
in Neural Information Processing
Systems, 30, 5998-6008.
https://papers.nips.cc/paper/7181-attention-is-all-you-need
Williams, A., Nangia, N., & Bowman, S. R.
(2018). A broad-coverage challenge
corpus for sentence understanding
through inference. In Proceedings of
the 2018 Conference of the North
American Chapter of the Association
for Computational Linguistics: Human
Language Technologies, Volume 1
(Long Papers) (pp. 1112-1122).
Association for Computational
Linguistics.
https://doi.org/10.18653/v1/N18-1101
:..........................................................................:
──[ EOF :: vol.3 / ghosts ]──────────────────────────────────────────────────//─
──