#letter frequency

[security-engineering]

Coincidence Index

The performed a coincide index on this cyphered text, aligning the first line with the second and so on, then counting how many letters were the same. Since Italian has a coincidence index of 1.94, if a coincidence index of an encrypted text is close to that value, the text is likely to be encrypted by a substitution cypher.

Each of the following texts has 420 character pairs.

The first text has 16 coincidences, giving it a coincidence index of 16/420*26 = 0.990.

The second text has 28 coincidences, almost double the first, giving it a coincidence index of 28/420*26 = 1.733.

We know that one of the texts used an enigma machine and the other used a mono-alphabetic substitution cypher. Since mono-alphabetic substitution cyphers preserve letter frequencies, it should have a coincidence index similar to the expected Italian coincidence index. As such the second text is using the substitution cypher.

[exxos-von-steamboldt]

(via Kgyw0tV.png (1493×1245))

[cmwindmuller]

100 random names (English Letter Frequency)

100 random 6-letter words. Starting letters are based on Relative frequencies of the first letters of a word in the English language and the following letters are based on How often does a given letter follow another in English?

Medare,Betheo,Blyzan,Mialed,Sioned,Austhr,Acupis,Avisst,Utitil,Areryz,Stherd,Wisoop,Antyzi,Farede,Biowha,Tedoon,Inengr,Angrer,Oecund,Thewir,Lfrdom,Swobea,Neding,Wedyze,Theean,Thotov,Thouss,Stevin,Yzerin,Ounorb,Touleg,Icthes,Whesor,Tyzedl,Rongnl,Taswan,Ndanca,Inange,Ctugan,Beenci,Perorr,Lecest,Gofoom,Yzedan,Spheli,Merowa,Dldyup,Terito,Oredsk,Dindll,Dgrten,Thouss,Ingles,Thedam,Atedee,Dleyze,Saryzi,Yqulde,Hassit,Tstryz,Besind,Epewil,Danond,Yzeanu,Isenel,Windyz,Traith,Asanth,Essyzz,Turtic,Aupait,Wargld,Busitu,Agefin,Ntrmio,Ykngnd,Pedrsp,Phisth,Ormeso,Chtere,Wonsha,Isheng,Usadrn,Thaded,Hofrel,Aneris,Sshamo,Horsir,Wadush,Pectur,Styzll,Usther,Oncord,Tideli,Femede,Anteay,Ofthab,Stedey,Stuged,Whethe

Blue is English letter frequency, Cyan is the generated letter frequency, and Magenta is overlap.

[paranoidbunbun]

So I did a thing...

I took all the words found at this place: http://www.mieliestronk.com/corncob_lowercase.txt

Which surmounts to about 58110 words.

Then I coded a small piece to get the letter fequencies and letter combinations. Mostly in order to create random words for my roleplaying group... I, err, may have gone a bit over board there.

Anyway!

I won’t show all the data here, cuss it’s 15.000 rows long, but I found some curious tidbits at least. Like the only double letter combination that ends words are ‘LZ‘ and ‘VS‘. There are others that almost always end words, of course, but they all can have at least another letter as well.

So let’s look at letters that start words!

-- first letter -- s: 0.1148 percent c: 0.0945 percent p: 0.0785 percent d: 0.065 percent r: 0.0625 percent a: 0.0599 percent b: 0.0552 percent m: 0.0507 percent t: 0.0496 percent i: 0.046 percent e: 0.0445 percent f: 0.044 percent h: 0.0349 percent u: 0.0331 percent l: 0.0316 percent g: 0.0316 percent w: 0.0265 percent o: 0.0239 percent n: 0.0158 percent v: 0.014 percent j: 0.0081 percent k: 0.0061 percent q: 0.005 percent y: 0.0025 percent z: 0.0015 percent x: 0.0002 percent

So yeah, the most common first letter is S with the least common X.

No love for X ;-;

So... what’s the point of all this?

None really... I did it because I wanted to be able to generate random words that actually are pronouncable. Which might’ve worked? I’ll let you be the judge of that!

Hled dess refenratings fies stroy taticellacch!

P.S If anyone wishes to know more about this, or is interested in a 227kb large file with the analysis, just ask me and I’ll be happy to talk more about it.

[datarep]

Letter Frequency Across Selected Languages

via Srirachachacha

Srirachachacha:

METHOD

I used Excel to plot the original chart, then tidied it up and assembled the data tables using Photoshop.

DATA

Spreadsheet of Data Used

SOURCES

Original Wiki Source

Specific Sources:

Mička, Pavel. "Letter frequency (English)". Algoritmy.net

Beutelspacher, Albrecht (2005). Kryptologie (7 ed.). Wiesbaden: Vieweg. p. 10. ISBN 3-8348-0014-7.

"CorpusDeThomasTempé". Retrieved 2007-06-15

Pratt, Fletcher (1942). Secret and Urgent: the Story of Codes and Ciphers. Garden City, N.Y.: Blue Ribbon Books. pp. 254–5.

Singh, Simon; Galli, Stefano (1999). Codici e Segreti (in Italian). Milano: Rizzoli. ISBN 978-8-817-86213-4.

[addictedtotype]

Letter Frequency Meter

http://www.sttmedia.com/characterfrequency-russian

http://practicalcryptography.com/cryptanalysis/letter-frequencies-various-languages/russian-letter-frequencies/

[themathematicspost-blog]

Enjoyed Playing Hangman: Here is something great for You

Here is some statistical information that is helpful when trying to solve cryptograms   Frequencies of the letters in the English language       

E12.51%

T9.25

A8.04

O7.60

N7.09

S6.54

R6.12

H5.49

L4.14

D3.99

C3.06

U2.71

M2.53

F2.30

P2.00

G1.96

W1.92

Y1.73

B1.54

V0.99

K0.67

X0.19

J0.16

Q0.11

Z0.09

The most common first letter in a word in order of frequency

          T, O, A, W, B, C, D, S, F, M, R, H, I, Y, E, G, L, N, O, U, J, K

The most common second letter in a word in order of frequency

          H, O, E, I, A, U, N, R, T

The most common third letter in a word in order of frequency

          E, S, A, R, N, I

The most common last letter in a word in order of frequency

          E, S, T, D, N, R, Y, F, L, O, G, H, A, K, M, P, U, W

More than half of all words end with

          E ,T, D, S

Letters most likely to follow E in order of frequency

          R,S,N,D

The most common digraphs on order of frequency

          TH, HE, AN, IN, ER, ON, RE, ED, ND, HA, AT, EN, ES, OF, NT, EA, TI, TO, IO, LE, IS, OU, AR, AS, DE, RT, VE

The most common trigraphs in order of frequency

          THE, AND, THA, ENT, ION, TIO, FOR, NDE, HAS, NCE, TIS, OFT, MEN

The most common double letters in order of frequency

          SS, EE, TT, FF, LL, MM, OO

The most common two-letter words in order of frequency

          of, to, in, it, is, be, as, at, so, we, he, by, or, on, do, if, me, my, up, an, go, no, us, am

The most common three-letter words in order of frequency

          the, and, for, are, but, not, you, all, any, can, had, her, was, one, our, out, day, get, has, him, his, how, man, new, now, old, see, two, way, who, boy, did, its, let, put, say, she, too, use

The most common four-letter words in order of frequency

          That, with, have, this, will, your, from, they, know, want, been, good, much, some, time, very, when, come, here, just, like, long, make, many, more, only, over, such, take, than, them, well, were

The most commonly used words in the English language in order of frequency

          The, of, and, to, in, a, is, that, be, it, by, are, for, was, as, he, with, on, his, at, which, but, from, has, this, will, one, have, not, were, or, all, their, an, I, there, been, many, more, so, when, had, may, today, who, would, time, we, about, after, dollars, if, my, other, some, them, being, its, no, only, over, very, you, into, most, than, they, day, even, made, out, first, great, must, these, can, days, every, found, general, her, here, last, new, now, people, public, said, since, still, such, through, under, up, war, well, where, while, years, before, between, country, debts, good, him, interest, large, like, make, our, take, upon, what

Related articles

A Better Strategy for Hangman (datagenetics.com)

[marionrobinson]

The Alphabet According to Saramago -poster.

Lettershapes have been created in NodeBox for a future animation. According to letter frequency in the original source text.