ᴅᴧτᴧ ᴘοᴙᴎ ѕтᴧᴙ

The intersection of #anonymous #leaks and #dataporn Solar sailing outer space. We resist. We make data porn. We are NOW(); United as 1. Divided by 0. Expect Us. Want to play around with data? Learn some python? Research background and context? Write articles based on data? Join Us! code // data Reference Network Visualization A interactive visualization of the references between diplomatic cables. v0.3 v0.2 v0.1 Clustered Sankey Arcs A work-in-progress visualization of the communication between diplomatic posts and U.S. departments. 2013-05-04 preview Cablegate Data A collection of data extracted from diplomatic cables. Cablegate NLP/NER JSON data from OpenCalais Named Entity Recognition. CableWeaving Stories woven out of cables by @TheRealDod contact @datapornstar dataporn || wlwardiary IRC chat.wikileaks-press.org

[soma-holiday-deactivated2020051]

Spread the datalove.

[dataporn]

Avatar

any suggestions for a new animated avatar?

[dataporn]

Cablegate Network Similarity n:n

Look mum, moar data pr0n

4000x4000px PNG

This visualization shows structural similarity between 4932 networks.

Each node is a network and each edge the structural similarity.

Some of these networks have identical structures.

The node size is the average weighted degree, or how similar the node is to it's surrounding peers.

How it is done

Load the Cablegate reference network.

Take the giant component and split it into smaller networks using community detection.

Take the remaining components.

Filter out networks with less then 10 nodes.

Create a B-Matrix portrait for each network.

Measure the matrix distance (or similarity) between each network (12.159,846 combinations).

Create a weighted undirected graph were each node is a network and each edge the similarity.

Use the distance measure as edge weight.

Remove edges with an distance measure of > 0.12 (0 means the network is equal).

Result: a graph of 4932 nodes and 19163 edges.

What does it mean?

We will look at the structures behind each node in detail in a future posting but from the visualization we can make some conclusions.

Islands of isolated structures with all networks being identical

A larger component with 8-10 different structures gradually shifting between similar structures.

Source code and more soon...

[dataporn]

Cablegate and Network Similarity

Browsing the visualizations generated from the cablegate reference network it becomes obvious that some formations of nodes and edges appear more often. The primary example is the star, often representing a action request from the state department and the related replies.

Network structure can represent a behavior, it can reflect how the references in diplomatic cables are used.

What common network structures exist and what do they represent?

To answer this question the complete graph has to be split down into smaller sub-graphs. Then each sub-graph has to be compared with each other for similarity.

However, even comparing two identical graphs is expensive.

A possible solution is introduced by J. Bagrow et al in the 2008 paper "Portraits of Complex Networks" that describes "large complex networks by introducing a new matrix structure, unique for a given network, which encodes structural information".

The graph is transformed into a two-dimensional array (B-Matrix) containing the distances between the nodes. A distance function compares two matrices and provides a normalized similarity measure.

A matlab/python implementation is available under the GPL so we adopted the code to igraph and converted the distance function from matlab to numpy.

(First time we worked with numpy. Any feedback / patches are more then welcome.)

Update 8th of Jul 2013: The interpretation is wrong. Enjoy three random graph's ^_^  (Less is better in the distance function.)

A quick first run looks promising. Here is a list of compared sub-graphs with a similarity measure greater then 0.7.

8 233 0.70518293711 12 233 0.704542921564 17 81 0.708275260907 18 81 0.700498352011 18 143 0.702374623427 20 81 0.71289338297 21 143 0.705450559671 27 81 0.700655200655 33 81 0.717296035877 34 143 0.710214144108 36 81 0.718753447325 49 81 0.706471610727 51 145 0.70617816092 51 233 0.725401960271 59 233 0.712744016078 60 81 0.709133610407 74 81 0.707651765916 81 17 0.708275260907 81 18 0.700498352011 81 20 0.71289338297 81 27 0.700655200655 81 33 0.717296035877 81 36 0.718753447325 81 49 0.706471610727 81 60 0.709133610407 81 74 0.707651765916

Visualization of sub-graph 17, 18 and 81 below, showing long path and circle structures in all three graphs.

[dataporn]

Manning

What's this? Nicholas Murphy Unclassified Stipulation, 117 Charged State Dept Cables

Not in Cablegate

10TOYKYO627 MAR 29 2010

10BRUSSELS382 MAR 30 2010

10GENEVA347 MAR 30 2010

10LIMA333 MAR 29 2010

10PRETORIA636 MAR 30 2010

10RABAT294 MAR 30 2010

In Cablegate

09STATE92655 SECRETARY'S MEETING WITH HONDURAN PRESIDENT ZELAYA

09STATE92657 JCIC-DIP-09-010: U.S. RESPONSE TO RUSSIAS START FORMAT 81 NOTIFICATION REGARDING INCOMPLETENESS OR INSUFFICIENT QUALITY OF THE RECORDING OF TELEMETRIC INFORMATION PROVIDED REGARDING THE TRIDENT II SLBM FLIGHT TEST OF DECEMBER 10, 2008

09TEGUCIGALPA891 TFH01: JUSTICE FOR MY ENEMIES: DE FACTO GOVERNMENT VIGOROUSLY PURSUES PROSECUTIONS AGAINST ZELAYA TEAM

09TEGUCIGALPA892 PRO-COUP BUSINESSMAN CLAIMS LECH WALESA PLANS VISIT

09SANTIAGO835 CHILE: Opposition Candidate Pinera Continues to Lead in Polls, But Faces Tight Second Round Against Frei

09SANTODOMINGO1017 SUPREME COURT CHIEF JUSTICE EMPHASIZES COMMITMENT TO JUDICIAL INDEPENDENCE AND COOPERATION ON EXTRADITIONS IN MEETING WITH CHARGE

09STATE92632 AFGHANISTAN TRANSIT - REQUEST FOR GENERAL AUTHORIZATION

09STATE92641 PERSUADING THE GOVERNMENT OF SRI LANKA ON FREEDOM OF MOVEMENT

09PRISTINA58 KOSOVO: CONTINUED PROGRESS, CHALLENGES AT KOSOVO PROPERTY AGENCY

09RIYADH1156 SAUDI MINISTER OF STATE MADANI ON UAE, YEMEN, SAUDI-ARMENIAN RELATIONS

09SANTIAGO831 Lagos on Honduras, Chilean elections

09SANTIAGO833 CHILEAN MFA DG IBARRA ON REGIONAL DEVELOPMENTS: RELATIONS WITH U.S. EXCELLENT, UNASUR STILL KEY FORUM

09LIMA1309 VRAE: GOP MOUNTS RESCUE OP FOR STRANDED SOLDIERS

09MEXICO2658 MEXICO ON UNRWA AT 60 - SPEAKING ROLE FOR ISRAEL

09NEWDELHI267 AMBASSADOR PRESSES MENON TO IMPLEMENT CIVIL NUCLEAR COOPERATION WITH U.S.

09PARIS217 FRENCH FEEDBACK ON KOSOVO RECOGNITION STRATEGY

09PRAGUE88 CZECH REPUBLIC: RESPONSE TO DEMARCHE ON DISCUSSION OF MEDVEDEV SECURITY PROPOSAL

09BOGOTA2873 TWO MEMBERS OF THE AFRO-COLOMBIAN CAUCUS ARRESTED FOR PARAMILITARY TIES

09BRASILIA1112 BRAZIL MORE SUPPORTIVE OF U.S. ARAB-ISRAELI NEGOTIATIONS APPROACH BUT NO AID COMMITMENTS YET

09BRASILIA1113 AUGUST 4-5 VISIT OF U.S. NATIONAL SECURITY ADVISOR TO BRAZIL

09CARACAS1168 CHAVEZ UPS THE ANTE: THE CRIMINALIZATION OF PROTEST

08KHARTOUM428 LIAR'S POKER: SUDANESE "CALL AND RAISE" ON USG NON-PAPER

09ADDISABABA1063 SCENESETTER FOR VISIT OF AMB. RICE TO ETHIOPIA

09BAGHDAD2390 MALIKI ADVISOR SAYS PM UNLIKELY TO JOIN SHI'A COALITION; FEARS IRAN WILL FOMENT VIOLENCE IN RUN-UP TO ELECTION

09BAMAKO85 IS THE REBELLION OVER? MALI BEATS BACK BAHANGA AND PREPARES FOR PEACE IN KIDAL

07VILNIUS13 LITHUANIA STRUGGLING TO MEET SCHENGEN REQUIREMENTS

08AMMAN535 AS NEW POLITICAL PARTY RULES LOOM, JORDANIAN POLITICIANS MANEUVER

08CAIRO569 ARAB LEAGUE'S HISHAM YOUSEF ON SUMMIT AND "YEMENI INITIATIVE"

08DARESSALAAM206 TANZANIA: MINISTER MEMBE BRIEFS AMBASSADOR GREEN ON COMOROS PLANS, ZIMBABWE ELECTIONS

08KHARTOUM246 UNAMID COMMANDER: BUILDING UP WHAT WE HAVE SHOULD BE THE PRIORITY

07USUNNEWYORK573 U.S. OFFICIALS BRIEF UNSYG ON PRESIDENT'S CLIMATE CHANGE INITIATIVE, DISCUSS DARFUR

07USUNNEWYORK575 UPDATE ON UN EFFORTS TO SET UP LEBANON TRIBUNAL

07USUNNEWYORK578 RUSSIA TRASHES MINIMALIST KOSOVO DRAFT RESOLUTION

07VIENTIANE12 REPRESENTATIVE MCCOLLUM MEETS NATIONAL ASSEMBLY LEADERS

07SANSALVADOR1375 EL SALVADOR: CONCURRENCE WITH 1206 FUNDING PROPOSALS

07SANTODOMINGO28 DOMINICANS FIGHT CORRUPTION - MUCH BARK, LITTLE BITE, SHOOTING SNAFU WITH JUSTICE OFFICIALS

07SUVA18 FIJI UPDATE 1/8/07: INTERIM GOVERNMENT MINISTERS; CHAUDHRY TO FINANCE; INTERNATIONAL INFLUENCES; APPARENT MURDER IN RFMF CUSTODY

07TUNIS47 SCENESETTER FOR VISIT OF DAS CARPENTER

07RANGOON22 THAN SHWE REPORTEDLY RETURNING TO BURMA

07REYKJAVIK203 Icelandic MFA announces inquiry into alleged CIA detainee flights

07RIYADH21 DEMARCHE DELIVERED: SECURING REGIONAL BORDERS AGAINST INFLUX OF FOREIGN FIGHTERS AND WEAPONS TO SOMALIA

07RIYADH22 FOLLOW UP ON AL MANAR AND AL AKSA TELEVISION; ARMS FOR LEBANESE ARMED FORCES

07RIYADH23 SAG STATES NO HIGH-LEVEL ENGAGEMENT OF SYRIAN OFFICIALS

07NEWDELHI80 CONGRESS GOVERNMENT IN HIMACHAL PRADESH PASSES A RELIGIOUS CONVERSION BAN

07PANAMA1197 PANAMA POST: EDITION V

07PANAMA1198 PANAMA: UIGHUR DEMARCHE DELIVERED TO 1ST VP AND FM

07PARIS4722 GERMAN DIPLOMAT AT FRENCH MFA DESCRIBES CAUTIOUS, CONSTRUCTIVE RELATIONS

07PARIS4723 FRANCE AND THE WTO AG BIOTECH CASE

07LAPAZ1949 OPPOSITION POLITICIAN PHYSICALLY ATTACKS INDIGENOUS LEADER

07LIMA2400 PERU FACES A "TORNADO OF PROTESTS"

07MINSK1024 AMBASSADOR VISITS HOSPITALIZED YOUTH LEADER FEDARUK

07MOSCOW5824 U.S., RUSSIA, AND EUROPEAN COOPERATION: FIRST WE NEED TO REMOVE OBSTACLES

07MOSCOW5825 RUSSIAN ELITES AHEAD OF RULING PARTY CONFERENCE

07DHAKA24 AMBASSADOR AND BRITISH HIGH COMMISSION MET WITH SHEIKH HASINA REGARDING MILITARY COUP

07KABUL68 DETAINEE LEGAL FRAMEWORK: GOA REPRESENTATIVES CONVENE TO DISCUSS TEST CASES, REVIEW BOARD

07KINGSTON25 JAMAICA: MALARIA UPDATE

07KUALALUMPUR40 MALAYSIA STILL WAVERING ON BURMA RESOLUTION

07LAGOS719 NIGERIA: NEW RIVERS GOVERNOR REINVENTS HIMSELF BUT FACES CHALLENGES IN TRIBUNAL

07BRIDGETOWN23 GRENADA TRYING TO READ ST. LUCIA TEA LEAVES

07BUENOSAIRES1341 DEPUTY FOREIGN MINISTER LOOKING FOR BILATERAL RE-ENGAGEMENT

07CARACAS2346 PDVSA CASH FLOW PROBLEMS

07CARACAS35 CHAVEZ PRESSING AHEAD WITH A SINGLE "REVOLUTIONARY" PARTY

07BEIJING152 CHINA URGES UNITED STATES TO BAR MEDIA, PUBLIC EVENTS FOR CHEN SHUI-BIAN TRANSIT STOPS

07BEIRUT1958 LEBANON: CHRISTIAN LEADERS DISCUSS BG HAJJ ASSASSINATION, PRESIDENTIAL ELECTION

07BOGOTA101 SCENESETTER FOR VISIT OF GENERAL PACE, CJCS

07BOGOTA5118 ON-AGAIN, OFF-AGAIN PEACE TALKS WITH ELN SET TO RESUME JULY 14

07BRATISLAVA665 SLOVAKIA: DCM REINFORCES KOSOVO, IRAN POINTS WITH DEPUTY FOREIGN MINISTER

07BAGHDAD63 IRAQ'S SUNNI VP DISCUSSES BAGHDAD SECURITY PLAN, MODERATE FRONT, SADDAM AND IRAN

07BAGHDAD64 COR FINANCE COMMITTEE CHAIR BEGINS 2007 BUDGET REVIEW

07BAGHDAD70 KURDISTAN PRESIDENT DISCUSSES OIL REVENUE SHARING, PRIVATE SECTOR DEVELOPMENT, RECONSTRUCTION AID

07BANGKOK111 AMBASSADOR PRESSES CNS SECRETARY GENERAL ON MARTIAL LAW

07BASRAH3 AN AMERICAN KIDNAPPED IN BASRAH

07BAGHDAD56 PRT TIKRIT: DOES THE CENTRAL GOVERNMENT'S REACH EXTEND TO SALAH AD DIN?

07BAGHDAD53 KURDISTAN PRESIDENT BARZANI DISCUSSES PESHMERGA, REGIONAL ADMINISTRATION AND MEDIA FREEDOM

07BAGHDAD42 MOI BOLANI SLOW TO DEVELOP REFORM PLAN

07BAGHDAD37 IRAQI POLITICAL LEADERS EMPHASIZE IMPORTANCE OF BAGHDAD SECURITY AND MODERATE FRONT, AT DINNER WITH DEFENSE SECRETARY GATES

07BAGHDAD36 AMBASSADOR'S JANUARY 5 MEETING WITH IRAQI PM ADVISOR AL-ASKARI

07BAGHDAD35 MALIKI'S GROWING WILLINGNESS TO TAKE ON JAM

07ASHGABAT1359 RUSSIAN DCM PLAYS UP GREAT GAME TO EXPLAIN U.S. INTERESTS IN TURKMENISTAN

07ANKARA2468 TURKEY: POLITICAL REFORMS STALLED AS GOT FOCUSES ON NEW CONSTITUTION

07ANKARA23 TURKEY: ERDOGAN IMPATIENT FOR U.S. ACTION AGAINST PKK

07ADDISABABA2197 (C-AL7-01035) ETHIOPIA: WFP OFFICIAL DISCUSSES ONLF AND COUNTERINSURGENCY IN OGADEN

06TRIPOLI648 LIBYAN CENTRAL BANK ESTABLISHING NEW TRAINING INSTITUTE

06TRIPOLI645 LIBYAN DEMARCHE ON CHINA SUMMIT WITH AFRICAN NATIONS

06TAIPEI3830 MTAG: UNSCR 1718 AND EXPORT CONTROL

06SUVA489 FIJI UPDATE 11/13: GCC TRIES TO MEDIATE; NO STABLE OUTCOME IN SIGHT

06SEOUL3885 PRESIDENTIAL CANDIDATE KOH GUN ON NORTH KOREA, ALLIANCE AND DOMESTIC POLITICS

06SEOUL3882 ROK TO ANNOUNCE POSITION ON PSI NOVEMBER 13, 2006

06RIYADH8811 UNPRECEDENTED MEETING BETWEEN USG AND SAG ON PETROLEUM INFRASTRUCTURE PROTECTION

06PRISTINA948 KOSOVO: PLANNING FOR POSSIBLE SERB REFUGEE MOVEMENTS POST-STATUS SHOWS GAPS IN ABILITY TO HANDLE LARGE MIGRATIONS

06PRISTINA947 KOSOVO: UNMIK GETTING OUT OF THE RETURNS BUSINESS

06MADRID2956 SPAIN: ACTIVE SUPPORT ON TURKEY/EU; WILL SEEK LEAST DAMAGING OPTION

06MADRID2955 SPAIN DOES NOT AGREE WITH KOSOVO TIMEFRAME

06KUWAIT4438 IRAQ DEBT FORGIVENESS: "A BLOODY, UPHILL BATTLE"

06KUWAIT4430 FREEDOM AGENDA: RUMBLINGS OF ANOTHER PARLIAMENT DISSOLUTION

06KATHMANDU3024 PM KOIRALA OPTIMISTIC ABOUT PROSPECTS FOR PEACE

06KATHMANDU3023 GETTING A SECURITY COUNCIL MANDATE FOR NEPAL'S PEACE PROCESS

06KABUL5435 PRT KHOST: NEW ABP COMMANDER OPTIMISTIC DESPITE DECIMATED/DEMORALIZED FORCE

06KABUL5421 PRT TARIN KOWT - GOVERNOR MONIB'S MILITIA

06KABUL5420 KARZAI ASKS QUESTIONS ABOUT ANP RANK REFORM

06COLOMBO1889 DEFENSE SECRETARY EXPRESSES CONCERN ABOUT POSSIBLE MAJOR LTTE ATTACK

06BELGRADE1681 ICTY ACTION PLAN SCORECARD, OCTOBER 2006

06BEIRUT3703 LAHOUD REJECTS PATRIARCH'S CALL TO QUIT

06BEIRUT3604 FORMER PM MIKATI INSISTS HE SUPPORTS SINIORA

06BEIRUT3603 UN OFFICIALS ON COORDINATION, TRIBUNAL, OVERFLIGHTS, HIZBALLAH

06BAGHDAD4205 SCIRI LEADERS BEMOAN LACK OF STRONG SUNNI PARTNERS IN IRAQI GOVERNMENT

06BAGHDAD2646 HUNGER STRIKE BY AD-DUJAYL DEFENDANTS

06ALGIERS1961 SKIKDA RAMPING UP REFINING, LNG CAPACITY WITH EYE ON EUROPE

05ALGIERS1836 DESPITE SOME EARLIER CONCERNS, GOA TAKING POSITIVE STANCE ON LUGAR MISSION AND ITS AFTERMATH

99ISLAMABAD495 PUBLIC DIPLOMACY: USAMA BIN LADIN

10REYKJAVIK13 LOOKING FOR ALTERNATIVES TO AN ICESAVE REFERENDUM

[dataporn]

117 cables Manning is charged with

10TOYKYO627 MAR 29 2010 10BRUSSELS382 MAR 30 2010 10GENEVA347 MAR 30 2010 10LIMA333 MAR 29 2010 10PRETORIA636 MAR 30 2010 10RABAT294 MAR 30 2010 09STATE92655 SEPT 4 2009 09STATE92657 SEPT 4 2009 09TEGUCIGALPA891 SEPT 4 2009 09TEGUCIGALPA892 SEPT 4 2009 09SANTIAGO835 SEPT 4 2009 09SANTODOMINGO1017 SEPT 4 2009 09STATE92632 SEPT 4 2009 09STATE92641 SEPT 4 2009 09PRISTINA58 FEB 12 2009 09RIYADH1156 SEPT 5 2009 09SANTIAGO831 SEPT 4 2009 09SANTIAGO833 SEPT 4 2009 09LIMA1309 SEPT 4 2009 09MEXICO2658 SEPT 4 2009 09NEWDELHI267 FEB 12 2009 09PARIS217 FEB 12 2009 09PRAGUE88 FEB 12 2009 09BAGOTA2873 SEPT 4 2009 09BRASILIA1112 SEPT 4 2009 09BRASILIA1113 SEPT 4 2009 09CARACAS1168 SEPT 4 2009 08KHARTOUM428 MAR 23, 2008 09ADDISABABA1063 MAY 7 2009 09BAGHDAD2390 SEPT 5 2009 09BAMAKO85 FEB 12 2009 07VILNIUS13 JAN 8 2007 08AMMAN535 FEB 20 2008 08CAIRO569 MAR 23 2008 08DARESSALAAM206 APR 1 2008 08KHARTOUM246 FEB 20 2008 07USUNNEWYORK573 JUL 13 2007 07USUNNEWYORK575 JULY 13 2007 07USUNNEWYORK578 JUL 13 2007 07VIETIANE12 JAN 8 2007 07SANSALVADOR13675 JUL 13 2007 07SANTODOMINGO28 JAN 8 2007 07SUVA18 JAN 7 2007 07TUNIS47 JAN 5 2007 07RANGOON22 JAN 8 2007 07REYKJAVIK203 JUL 13 2007 07RIYADH21 JAN 8 2007 07RIYADH22 JAN 8 2007 07RIYADH23 JAN 8 2007 07NEWDELHI80 JAN 8 2007 07PANAMA1197 JULY 2007 07PANAMA1198 JUL 13 2007 07PARIS4722 DEC 14 2007 07PARIS4723 DEC 14 2007 07LAPAZ1949 JUL 13 2007 07LIMA2400 JUL 13 2007 07MINSK1024 DEC 14 2007 07MOSCOW5824 DEC 14 2007 07MOSCOW5825 DEC 14 2007 07DHAKA24 JAN 7 2007 07KABUL68 JAN 7 2007 07KINGSTON25 JAN 5 2007 07KULALUMPUR40 JAN 8 2007 07LAGOS719 NOV 1 2007 07BRIDGETOWN23 JAN 5 2007 07BUENOSAIRES1341 JUL 13 2007 07CARACAS2346 DEC 14 2007 07CARACAS35 JAN 5 2007 07BEIJING152 JAN 8 2007 07BEIRUT1958 DEC 14 2007 07BOGOTA101 JAN 5 2007 07BOGOTA5118 JUL 13 2007 07BRATISLAVA665 DEC 14 2007 07BAGHDAD63 JAN 7 2007 07BAGHDAD64 JAN 7 2007 07BAGHDAD70 JAN 8 2007 07BANGKOK111 JAN 8 2007 07BASRAH3 JAN 5 2007 07BAGHDAD56 JAN 7 2007 07BAGHDAD53 JAN 7 2007 07BAGHDAD42 JANUARY 7, 2007 07BAGHDAD37 JANUARY 5 2007 07BAGHDAD36 JAN 5 2007 07BAGHDAD35 JAN 5 2007 07ASHGABAT1359 DECEMBER 14 2007 07ANKARA2468 OCTOBER 4 2007 07ANKARA23 JAN 8 2007 07ADDISABABA2197 JULY 13 2007 06TRIPOLI648 NOV 10 2006 06TRIPOLI645 NOV 10 2006 06TAIPEI3830 NOV 12 2006 06SUVA489 NOV 12 2006 06SEOUL3885 NOV 12 2006 06SEOUL3882 NOV 10 2006 06RIYADH8811 NOV 11 2006 06PRISTINA948 NOV 11 2006 06PRISTINA947 NOV 11 2006 06MADRID2956 NOV 27 2006 06MADRID2955 NOV 27 2006 06KUWAIT4438 NOV 12 2006 06KUWAIT4430 NOV 10 2006 06KATHMANDU3024 NOV 10 2006 06KATHMANDU3023 NOV 10 2006 06KABUL5435 NOV 12, 2006 06KABUL5421 NOV 10, 2006 06KABUL5420 NOV 10 2006 06COLOMBO1889 NOV 10 2006 06BELGRADE1681 OCT 17 2006 06BEIRUT3703 NOV 27 2006 06BEIRUT3604 NOV 12 2006 06BEIRUT3603 NOV 11 2006 06BAGHDAD4205 NOV 10 2006 06BAGHDAD2646 Jul 24, 2006 06ALGIERS1961 Nov 12, 2006 05ALGIERS1836 Aug 29 2005 99ISLAMABAD495 Jan 25 1999 10REYKJYAVIK13 January 10, 2013

[dataporn]

OffshoreLeaks Graph

The ICIJ dropped some graph data as .csv file containing different types of financial "objects".

We processed the data with igraph, extracted the giant component, run multi-level community detection on the giant and rendered a layout with Gephi's OpenOrd plugin.

Graph properties:

full: 2.004,268 nodes 1.049,070 edges giant: 245,914 nodes 925,382 edges communities: 4448

node color: community

node color: community, curved edges

Download GraphML file with position data and high resolution 4096x4096 PNG files..

[dataporn]

Can't really figure out what degree of ownership I feel/have over my data, or myself as a NUMBER, not a person.

Molly Buckley

[dataporn]

Mr. Z

A unusually vague cable (1975STATE247517) from Lawrence Eagleburger to Henry Kissinger on the 17th of October 1975 describes a conversation with Mr. Z about a program and a continues leaking process and how Kissinger should give names within categories and that certain lawyers should talk with each other.

The first mention of Mr. Z leaves a specific amount of space around the letter "Z" indicating the position of the letter and the length of the actual name.

AS I TOLD YOU HE WOULD, MR. Z CAME IN

1975STATE247517 also mentions a "Maw" who turns out to be Carl Maw, Kissinger's private Lawyer who became a Under-Secretary at the State Department.

A second cable (1975STATE249558) a few days later on the 21st of October 1975 mentions both Mr. Z and Carl Maw and interestingly a New York Times article by political columnist William Safire.

Conveniently Eagleburger provides the totally vicious article in another cable (1975STATE248575) to Kissinger, with the comment that he would pursue the subject gently with Ron.

Ron referes to Ron Ziegler who seems to be the person behind "Mr. Z". He remained a close associate to Richard Nixon despite the Watergate scandal.

The Safire article is a interesting read and we would like to find out more about the meaning of the program, the categories and the leaking process, but we need your help with that!

Join us digging the cables! We need more contributors! Our contact details are in the sidebar on the left ^_^

[dataporn]

Allen Campbell

Digging the cables we found this TAGS search with the first two cables behind FOIA-wall and the others only note that "REFTEL HAS BEEN RAISED TO EXDIS". Interesting. ^_^

A search for Allen Campbell Rhodesia find's this:

After leaving The Corps, Campbell traveled throughout Eastern and Western Africa including work in Rhodesia in 1973, and flying humanitarian operations in Mozambique in 1984. Campbell was asked to assist the American Embassy in Kampala, Uganda to fly missions exploring reports of human bodies in Lake Victoria. He subsequently ran multiple rescue missions at the height of genocidal massacres in Rwanda. Campbell granted this interview to serve as a witness to the horrors of the violence in Rwanda, as well as to continue his process of recovery from extreme PTSD suffered as result of his involvement.

...and a face.

source

Rhodesian in 1973 was in the middle of the Bush War.

Who can find more information?

[dataporn]

[dataporn]

NARA cables: indications of systematic removal

Currently 706,740 NARA cables are only available digitally as metadata. The main text content of these cables lies hidden behind a FOIA request.

Using the text search we can look for patterns that reveal the decisions made behind this selective digital record. What is hidden and why?

In the search WikiLeaks made it possible to filter the cables by Origin. This is not just the name of the Embassy or the Department of State but can be a specific Office or Department. A list of known origins is also available.

Filtering by Origin we can clearly see some patterns.

(First number amount listed cables. Second number amount of available cables)

1,534 / 1: Bureau of Politico / Military Affairs 2,519 / 7: Bureau of African Affairs 8,388 / 0: Bureau of Public Affairs 1,892 / 0: Bureau of Educational and Cultural Affairs 1,858 / 1: Bureau of International Organization Affairs 761 / 1: Bureau of Oceans and International Environmental and Scientific Affairs/Ocean and Fishery Affairs 15,094 2,736 / 0: Office of the Chief of Protocol 2,317 / 0: Office of International Conferences 2,451 / 0: Office of Personnel 213 / 0: Office of Security 7,717 1,566 / 0: Passport Services 350 / 0: Foreign Affairs Document and Reference Center 5442 / 12: National Archives and Records Administration 282 / 0: Pan American Health Organization Organisation for Economic Co-operation and Development (Paris) Salt Lake City Talks (MBFR) 2,198 498 / 6 Department of Commerce 278 / 0 Department of Health Education and Welfare 671 / 0 Department of Humanitarian Affairs 4,446 / 3 Department of State Bureau of East Asian and Pacific Affairs 8,802 / 3 Department of State Bureau of European and Eurasian Affairs 1,615 / 1 Department of State Bureau of Intelligence and Research 4,269 / 17 Department of State Bureau of Near Eastern Affairs 741 /487 Department of State Secretary of State Aboard Aircraft 21,320

Person as origin: all cables are behind the FOIA wall.

1,484 Schweiker, Richard Schultz 414 Bayh, Birch 349 Beal, J Glenn Jr 219 Bennett, Charles E 498 Bentsen, Lloyd 981 Borg, C Arthur 322 Brooke, Edward W 1,236 Buckley, James L 471 Burgess, Harrison W 588 Case, Clifford P 412 Chiles, Lawton 308 Church, Frank 300 Clark, Dick 1,590 Cranston, Alan M. 520 Davis, Jeanne W 301 Eagleton, Thomas F 200 Fascell, Dante B 518 Ford, Gerald R 232 Fraser, Donald M 334 Glenn, John 296 Goldwater, Barry 662 Griffin, Robert P 257 Habib, Philip C 522 Hart, Philip A 789 Hartman, Arthur Adair 390 Hatfield, Mark O 242 Horton, Frank 262 Hummel, Arthur W Jr 623 Humphrey, Hubert H 1,013 Ingersoll, Robert S 463 Jackson, Henry M. 642 Jackson, Murray E. 2,496 Javits, Jacob K. 523 Jones, Adolph W 1,624 Kennedy, Edward M. 1,425 Kissinger, Henry 300 Leigh, Monroe 267 Long, Russell 610 MCCORD, FRANKLIN O 377 Magnuson, Warren G 399 Malmborg, Knute E 408 Mansfield, Mike 295 Mathias, Charles Mcc Jr 281 McCloskey, Robert J 322 Mondale, Walter F 223 Morgan, Thomas E 203 Nelson, Gaylord 213 Packwood, Robert W 586 Percy, Charles H 288 Proxmire, William 369 Robinson, Charles W 399 Rodino, Peter W Jr 223 Rogers, Paul G 294 Rogers, William D 1,484 Schweiker, Richard Schultz 641 Scott, Hugh 221 Scott, William L 307 Sisco, Joseph J 474 Sparkman, John Jackson 5,384 Springsteen, George S 817 Stevenson, Adlai Ewing 244 Symington, Stuart 223 Taft, Robert Jr 390 Talmadge, Herman E 283 Thurmond, Strom 472 Tower, John G 1,675 Tunney, John Varick 205 Weicker, Lowell P Jr 622 Williams, Harrison A JR 43,005 total sum: 89,334 total hidden: 706,740

Redacted Cables

A last finding. NARA marked 156 manually redacted cables with "Sensitive Information Deleted, NARA 2/23/2006".

[dataporn]

Removed NARA cables

The NARA cables made accessible by WikiLeaks contain 151,350 almost empty records. Many however still contain a subject and can be referenced by other cables.

Here is a list of keywords for exclusion to get that list down to ~8,000 hits.

-FUGITIVE -AMCIT -WAIVER -PLEX -REPATRIATION -ALIEN -PARAKEET -RHODESIANS -SPLEX -VISAS -VISA -NIV -"IV CASE" -PASSPORT -BOULDER -"NAME CHECK" -NAMECHECK -PAROLE -"n-a" -"W-W" -"IR CASE" -"IV CASES" -"DISPOSITION OF REMAINS" -"NI CASE" -EXTRADITION -REFUGEE -DEPORTATION

[dataporn]

We run the OpenCalais named entity extraction on all cables from #cablegate. Result: ~1.5G compressed .7z archives, extracted ~19G of .json data.

Use the data and share the results!

It took 10 days to process the cables so keep the torrent alive and seed. No seeds? contact @datapornstar.

[dataporn]

For more then a year we worked on cablegate and extracted specific subset's of data. This is a collection of links to find them.

[dataporn]

diplomatic chaos

[antoine-crenn-deactivated201612]

[dataporn]

Internet Rev11  1*23. Linogravure 2011