What are you looking at? @markgamache-blog - Tumblr Blog

Need for EC2+EBS Speed

Because of my love/hate relationship with Amazon's EBS I started looking for alternatives. One interesting offering is from Zadara. They have a direct connect to Amazon and offer iSCSI network storage backed by SSD drives. You have to operate within a VPC to use the service, but it took me only a few minutes to set one up (and I suck at admin stuff). I would be surprised if Amazon doesn't offer this type of service in the future. Zadara isn't perfect, but if you are struggling with disk IO it could be useful.

C# List<> is not thread safe

You need to use ConcurrentBag ( var foo= new ConcurrentBag<string>();) instead of List<string>. I had no idea..

I found this out by using a loop in my code that was not adding all the data to a List

var ids = new List<string>();

Parallel.ForEach(docs, doc =>

{

ids.Add(doc.Id.ToString());

});

//docs contains 1000 entries but the ids variable would always have < 1000 entries.

Classic Windows dialog. I think I should update.

#Windows dialog

I think there's about 17 million visitors a year to Disneyworld. So that's approx 50K a day. You quoted 1 million a day. Not sure where you go those numbers, but it may be an error.

I've seen figures from 24-42 million per year for Disney World. On Wednesday of last week park staff said they were getting ready to close down the park to additional visitors. They also said that represented 1 million visitors (that number does seems high in retrospect).

Vacations and Fingerprint Scans

I lobbied against SOPA and PIPA by writing a letter to my representatives (via EFF's service), but I am not what you would call an activist. After years of promising my wife and children a *real* vacation (not just visiting relatives) I took them on an all-american vacation to Florida. We visited the beach, Kennedy Space Center, SeaWorld and Disney World. For those contemplating such a trip, be warned, SeaWorld or Disney World will scan your fingerprint when you enter the parks. I think they also do this at other large theme parks, but I don't have firsthand knowledge. I was caught off-guard the first time (at SeaWorld) and complied with the request. After that (at Disney) I requested a no-Fingerprint admission. This is possible, even easy, but they don't advertise the option. You just need to provide a valid photo id. My wife, of course, was mortified the first few times I made a scene and called attention to myself. But what the hell do they need my thumbprint for? Even more alarming (to me) is that my eleven year old son was also scanned at Disney. Am I right to be concerned? Anyway, I am posting this as a warning to those who might go to Florida (or other places), you might be faced with a privacy dilemma.

PS. It dawned on me after reading HN comments that a website can not save any data from children (see COPPA), but a park can scan and save fingerprint and other bio-metric data without any oversight or rules.

Google complies with 88% of government data requests. Do you think Disney would be any different? Disney could even be sharing data with DHS, how would we know? Maybe it's tin-foil-hat thinking, but there's no oversight or accountability. Millions of people visit Disney each year. That's a big honeypot (with mouse ears on it).

The Death of College

Colleges are fundamentally a certification system. You pay money to acquire knowledge and take tests to assure your knowledge acquisition. After gaining enough knowledge in a particular domain (and enough general knowledge) you are certified with a 'degree'. Some universities have a better certification than others (Ivy League vs Ivy Tech), but that's just a branding exercise. This system has two problems:

The first problem is that college is shaky investment theses days. The college system is built on a solid return on investment for your time and money. Read the Wikipedia page on the 'education bubble' if you are desperate for some sleep.

The real problem for colleges is *not* rising costs (they could probably deal with that so some degree). It's that access to knowledge is pretty damn easy these days. This isn't the middle ages, you don't need to travel to far off cities and visit the handful of men who hold the keys to understanding the world.

It's pretty well known now, but if you haven't seen Khan Academy it's worth a look. Kahn's simple presentation style and effective delivery make me want to cry. I could have spent months, not years learning much of what I know.

This is the future of colleges in America. Stripping the college proposition to it's core: College Degree, No Class Time Required. Get your degree by showing what you know.

EC2 Provisioned IOPS & Windows Surprising Results

So i checked out the new EC2 Provisioned IO instances to see how well they performed. Specifically I wanted to know if you could get more speed by using four disks in a stripped array. What I found really surprised me....

IOPS Machine: m1.large Windows 2008 us-east-1b

Single Disk (100 gb / 1000 IOPS):

Sequential Write 21 MB/s

Sequential Read 40.3 MB/s

Random Read 40.3 MB/s

Stripped (4x 100 gb / 1000 IOPS)

Sequental Write 33.4 MB/s

Seqential Read 59.1 MB/s

Random Read 55.2 MB/s

Non IOPS machine m1.large Windows 2008 us-east-1c

Sequental Write 25.2 MB/s

Seqential Read 47.4 MB/s

Random Read 51.0 MB/s

Method: Using DiskTT (Windows Cache disabled) file size of 4GB and block size of 64kb. Both machines use the same AMI.

That's right: the non IOPS us-east-1c machine is really faster then the east-1b provisioned machine. Amazon implies that the infrastructure in each data-center is equivalent, but clearly this is not the case.

I was able to get a 37% bump in random read speed from the stripped array. Not life changing but it's a significant performance improvement.

#windows ec2 #IOPS #AWS #Amazon #ec2 #windows #2008 #EBS #stripping #benchmark

Mobile-Phone Location Data Not ‘Constitutionally Protected’

The government is arguing that your phones' location is not protected under the 4th amendment. Here's the problem: The argument invokes what are called “third-party records”. If your data is stored by a third party (cell companies in this case) you lose the expectation that it is private. The government can get the data without probable cause and without a warrant. From the gov: "no reasonable expectation of privacy exists in the routine business records". In the era of cloud computing what constitutes routine business records? When all of our private data is stored by third parties like Mint, Dropbox, Google, Apple etc... What becomes routine business records and what is not? It requires a warrant to use a GPS to track a car, but tracking via phone info is legal? Again the Gov: “But when the government merely compels a third-party service provider to produce routine business records in its custody, no physical intrusion occurs”.

The world is changing and the courts are struggling to keep up.

#4th amendment #privacy #government #cell phones #courts

Better Windows Monitoring with CloudWatch

The built in monitors for AWS CloudWatch are limited. I needed to watch critical things like disk space and memory utilization to let me know that my machines are in danger before my apps are offline. CloudWatch is flexible and lets you report on anything you like (using the AWS API) so I could write something myself. I thought there had to be something canned to do this, right? There is, Eleven41's CloudWatch Monitor for Windows. (MIT License)

Just download and unzip to a directory

Set your security creds in the CloudWatchMonitor.exe.config file

execute CloudWatchMonitor.exe -i as an admin to install as a service

Start the service

It has all the basic metrics that you want for disk and memory usage. Note: I don't know how to get these to appear in the Management Console, but you can use the web interface to set alerts if the number go out of whatever range you set.

#aws #EC2 #wiNDOWS #CLOUDWATCH #api

Twitter Bootstrap & .net MVC 4

Twitter Bootstrap is a really cool lightweight library to enable modern looking responsive web development. It's pretty easy to incorporate with .net MVC 4, but there are two items you need to do when using MVC 4 with Twitter Bootstrap:

Update jquery to the version that Bootstrap needs (currently 1.7.1) you'll get JS errors if you don't. MVC 4 is currently at 1.6.x.

Edit your Global.asax file to enable BundleTable.Bundles.EnableDefaultBundles(); and disable BundleTable.Bundles.RegisterTemplateBundles(); or your JS bundle won't include the bootstrap.js (if you include it in your project) more info

MS-SQL | how to create a fast GUID index

Need to use a GUID for a row identifier (identity column)?

For any DBA reading, this is probably old hat, but MS-SQL server 2005 introduced a new command for creating GUIDs= NEWSEQUENTIALID(). This allows a fast index based on a GUID key that performs close to the speed of an integer index. Just use the command NEWSEQUENTIALID as the default in the identity column of your table and viola, you have a very fast GUID index.

note: If privacy is a concern, do not use this function. It is possible to guess the value of the next generated GUID and, therefore, access data associated with that GUID (if the GUID is exposed).

msdn docs

The lazy way to multi-thread loops in .net (4.0)

Here's the easy way to turn foreach loops into multithreaded execution. Using the Task Parallel Library. TPL manages the threads and semaphores and all that stuff that stuff so hacks like me can make it work well. Note the use of a lambda inside the loop. This is actually executed in another thread so careful that you don't reference variables out of scope. Here's sample code:

using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; namespace MultiThreadingSample { class Program { static void Main(string[] args) { var sampleList = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }; Parallel.ForEach(sampleList, item =>{ Console.WriteLine("Thread start" + item.ToString()); Thread.Sleep(1000); Console.WriteLine("Thread end" + item.ToString()); }); Console.WriteLine("done with all threads..."); Console.ReadLine(); } } }

Want to control the number of threads? Use this:

var parallelOptions = new ParallelOptions(); parallelOptions.MaxDegreeOfParallelism = 5;//or whatever you want Parallel.ForEach(sampleList, parallelOptions item =>

#.net (4.0)#Task Parallel Library #threads

The nuts and bolts of optimizing .net MVC 3 routes

From the Stack Overflow guys (who know their shit)

Here are four things to look at:

The Regex cache - only holds 15 so if you have more routes then that it's an issue (if you use regex matching on your routes)

Stop using string based route constraints (build them in code -- see article)

Put most used routes on top

Disable file checks for static content

#.net MVC 3 #routes

Elasticsearch as nosql

If you are looking for a fast nosql database don't overlook Elasticsearch. Props to Andrew Doran who recommended it. Like SOLR it is based on Lucene but it's what SOLR should have evolved into, but didn't. It has JSON-REST and Memcache interfaces and unlike SOLR it stores all of your documents in the original form (this is a big deal - trust me). Built-in support for shards and load balancing with auto-discovery (yes even on Ec2). It can work as a key-value store or full search capability. There is a Windows service installer for those of you like me working in .net and Windows. I also recommend the .net client nest it's not complete yet, but it's the best of the pack (for now).

#Elasticsearch #nosql

This is what pays the bills. It's boring, but unencumbered by project managers and (too much) reporting. Any mistakes are all mine no one else to blame.

P3P Privacy files and IE iframes

So IE blocks cookies in iframes that don't a have a policy file set. How do you fix this?

create a policy with the IBM editor (I hate this domain, but it's the only valid source for the editor) http://www.softpedia.com/get/Security/Security-Related/P3P-Policy-Editor.shtml

Add a header to your pages or though a .htaccess file (or custom http headers in IIS)

the header? key = "P3P value= "policyref="/p3p/p3p.xml", CP="ALL DSP COR NID TAIa OUR NOR STA""

Note: The P3P a legally binding document so you might want to read up on what you are getting into.

Trending Blogs

Recently Viewed Blogs

What are you looking at?