#windowsazure

[webtechexperts]

What is Microsoft Azure? Notably, #MicrosoftAzure, formerly known as #WindowsAzure, is #Microsoft’s public #cloudcomputingplatform. It provides a range of #cloudservices, including #compute, #analytics, #storage, and #networking. Read and learn more here: https://josephmuciraexclusives.com/microsoft-azure/ https://www.instagram.com/p/B-4yz6glCAE/?igshid=1i55x4kt3lhx0

[vervesystems]

Windows Azure is a platform built for cloud computing and its flexibility assists to quickly develop, install and manage applications through a worldwide link of Microsoft data centers.

[santexgroup-blog]

A WINDOWS AZURE SCREENSHOT (MICROSOFT CLOUD COMPUTING)

Automated: You can easily manage and automate almost everything they provide. From azure SQL backups to computing processing.

Usage based: You pay only for what you use.

Elastic: You can easily scale according your needs.

Managed resources: You can administrate all you have from your control panel anywhere.

Microsoft keeps all these services replicated in bigs datacenters like this one:

Actually Microsoft adds new datacenters from time to time. Since July 2014 is available a new Microsoft Azure datacenter in the southern region of Brazil (São Paulo). This is great news and means we have the first Microsoft Azure datacenter in Latin America where we can host our services.

Do you want to know more about this? Visit us here

[onlinesoftwaretrainings]

Special Offer Rs.12,000/- Only (Online Traning) Windows Azure Online Training on 3rd Jan @ 7:00 AM (IST) By Real-Time Expert Demo link: https://zoom.us/j/294129816 📧: [email protected] | 📲: [phone] 💥Features of Online Training: ☑️ Real-Time Oriented Training ☑️ Live Training Sessions ☑️ Softcopy of Materials ☑️ Interview Preparation Tips ☑️ FAQ's

[idese]

Administración de Servicios de Internet y Desarrollo de soluciones Microsoft Azure

Curso subvencionado al 100% por el Ministerio de Empleo y Seguridad Social Garantía Juvenil. en Valencia

Al finalizar el curso el alumno será capaz de:

Instalar, configurar, administrar y mantener servicios comunes de provisión e intercambio de información utilizando los recursos de comunicaciones que ofrece Internet.

http://www.idese.es/home/formacion-gratuita.html?task=view_event&event_id=694

[davidpapkin]

(via https://www.youtube.com/watch?v=i3E2ujZ9TPQ)

[revelcloud]

Windows Azure - Disk Performance Surprises Explained

8A few months ago, we blogged about some surprising metrics coming from Azure Disks.  Notably, we found instances where Standard Storage was faster than Premium Storage and instances where Premium Storage was dreadfully slow (as in 2MB/s).  A great guy from the Azure storage team reached out and helped me to understand better what was going on.  

Here’s my take on what’s going on.  Caveat: I’m not a storage expert, just a guy trying to make sense of this stuff.

First off - when measuring the disk throughput, it is going to be governed by the “transfer request size” and the amount of concurrency (”number of threads” in CrystalDiskMark or “Outstanding I/Os” in Iometer).  Azure disks (and likely all cloud disks) are more heavily dependent on request size & concurrency to achieve high throughput than would be a local drive.  We’ll see how that plays out below.8

Before we update the benchmarks, let’s update/explain some of our findings from the original post.

In the initial post, CrystalDiskMark showed a dreadful 2.207MB/s write speed for the Premium Disk using a 4k transfer request size and 1 thread (this is the bottom right cell of the test) whereas the Standard Disk showed 61.82.  The reason for this is that the Standard Disk was also the o/s disk and this disk has read/write caching enabled - so it was the write cache that “sped up” this test.  As of yet, you cannot enable the write cache on data disks and using the write cache does expose some risk of data lost.  So, the conclusion that the Standard Disk is faster in some cases than the Premium Disk is flawed - it’s the “Standard Disk with read/write caching” that may be faster.

Regardless of #1 above, we’re still left with the question: “Why does Premium Storage only deliver 2.207 MB/s write speed especially when local SSD is 40x faster (82.75MB/s) for the same test?”  The answer, turns out to be pretty straightforward.  The cloud drive is limited to 500 IOPS per thread - so with a 4K transfer size and 1 thread, you get about 2MB/s (4k * 500/s = 2MB/s).  If you change this to 2 threads, you get 4MB/s.  This is the most simple example of how concurrency affects cloud drives.  You could also get 4MB/s by doubling the transfer size from 4k to 8k.  

Now, let’s update some benchmarks using Iometer (the Azure storage expert showed me this tool which provides more explicit control over transfer size and concurrency.  

I’m going to use a 4k transfer size, as this is my best guess as to what is likely for ElasticSearch (I’m still very unclear on how transfer size is affected in “real life” between the application stack and the o/s).  This corresponds to the built in “4 KiB; 0% Read; 0% random” ‘Access Specification’ in Iometer - with 0% Read, I believe this means it’s a write-only test.  When I use the term “threads” below, it corresponds to the “# of Outstanding I/Os” in Iometer.

I’ll just provide the numbers for the Premium Disk (P30 disk on DS12 Azure VM - 4 cores, 28GB RAM) compared to local SSD (4x 800GB SSD in Raid 10 - 6 cores, 64GB RAM).  I realize the machine sizes are different and may skew the results (but hey, you’ve got to work with what you’ve got!).

The maximum IOPS for the DS12 (12,800) and the P30 disk (5,000) are taken from this article as are the maximum throughput for the DS12 (128MB/s) and the P30 (200MB/s).

Premium Storage - 4k, 1 Thread

Notes: the highlighted portions show the key metrics - IOPS (474) and Throughput (1.94MB/s).  This is exactly what we expect - a single thread is limited to 500 IOPS and with a transfer size of 4k, we get the expected throughput (4k * 500 IOPS = 2MB/s).

Premium Storage - 4k, 8 Threads

Notes: again, we get the expected result.  IOPS goes to ~4,000 (8 threads * 500 / thread = 4,000) and throughput scales accordingly.

Premium Storage - 4k, 16 Threads

Notes: here we get something interesting as we see that the IOPS is not 16 * 500 = 8,000 but rather 5,000 which is the limit for a P30 disk.  We can conclude here that: The maximum throughput on a single P30 disk with a 4k transfer size is 20MB/s.  This is not a limit of the VM, but rather the disk.  In theory, you could stripe 3 P30 disks on a DS12 and increase the max IOPS to 12,800 which is the limit for the VM.

Premium Storage - 32k, 16 Threads

For kicks, I ran this test which has a very high transfer size (IMO) and a very high degree of concurrency (16 threads).  But, this does provide the maximum throughput as advertised on the DS12 (128MB/s).

Notes: here you can see that while we haven’t reached the maximum IOPS for the P30 disk (5,000), we have reached the maximum throughput for the DS12 (128MB/s).

Local SSD - 4k, 1 Thread

Local SSD - 4K, 8 Threads

Local SSD - 4K, 16 Threads

Notes - Looks like we’re getting close to the max IOPS for this disk array.  Even when I move to 64 threads, it only drives IOPS to 26,000 (minor increase from the 24,685 in the 16 thread test).

Local SSD - 32k, 16 Threads

Notes - What?  1GB/s?  Now that’s absolutely blazing fast!

---------------------------------------------------------------

Conclusions:

This is a pretty complex scenario, dependent on #cores, Windows, ElasticSearch, Lucene, Java, document size etc.  I feel pretty good that they capture a “typical” ElasticSearch scenario, but obviously there are a lot of variables involved.  Regardless, for this test setup, bastardized though it may be, I feel OK in making this conclusion - and it matches my ‘empirical’ evidence in our applications having run on both DS12|Premium-Storage and physical machines with Local SSD.

SSD is significantly faster (5x to 30x) than Premium Storage, regardless of transfer size or concurrency.  For “typical” ElasticSearch scenarios (see below) SSD is 3x to 5x faster than Premium Storage

“Typical write” ElasticSearch scenario (4k transfer, 8 threads), SSD is 5x faster than Premium Storage (92MB/s vs 15MB/s).  By default, ElasticSearch uses #processers as the number of bulk operation threads - this means only 4 threads on the DS12; a good optimization if running ElasticSearch on a DS12 may be to increase many or of all the threadpools.

“Typical read” ElasticSearch scenario (4k transfer, 8 threads), SSD is 3x faster than Premium Storage (281 MB/s vs 67MB/s).  [These tests are not shown above.]

Of course, everything is based on particular scenarios and workloads - there can be no steadfast “rules” about what environment is optimal for every scenario.  Disclaimer aside, I hope this helps expand the understanding of disk speed, both on and off Azure.

[revelcloud]

Why we moved away from Windows Azure

It wasn’t what we wanted, and it certainly wasn’t easy, but today, we can confirm that we’ve moved all of our apps from Windows Azure.  This post explains why, after 4+ years on Azure, we’re moving on.

Our setup

We run a pretty simple setup - a number of web apps that pull data from ElasticSearch and get it back to the client.   Our ElasticSearch data spans about 1TB - so more that a few records but not “big data” by any stretch.  From April 2011 to April 2015, we ran on “Cloud Services” (aka Platform as a Service).  In April 2015 we moved to “Virtual Machines” (aka Infrastructure as a Service).  At the end (August 2015) we were running a single Web App with Auto Scale, connected to two DS12 VMs.  Each VM had 4 cores, 28GB RAM and used the “P30 Premium Storage” disk for Elastic Search.

Issue #1: PSS turned out to be second class

We bet on Platform as a Service.  It sounded great (someone else manages the hardware and the o/s) but eventually it just fizzled out.  It received very little updates and the “Cloud Drive” system that allowed you to mount a drive was deprecated, and the SSD-based premium storage that we wanted for ElasticSearch wasn’t made available to PSS - this is the primary reason why we moved to VMs in April 2015.  

As an aside, the PSS replacement which is now called “Web Apps” is solid (tons of updates and innovation) but this is for the web tier only (i.e. not ElasticSearch).

Issue #2: Storage/Disk performance

Premium Storage is confusing - the performance is based both on the machine and the disk (see here for details).  For the D12s I was using, it maxes out at 150MB/s - in theory.  In practice, we found out later that Premium Storage may be slower for writes than Standard Storage and it’s considerably slower than local SSD.  We posted details and benchmarks from CrystalDiskMark here.

That any cloud storage is slower than SSD is expected and forgivable - but that Premium Storage ended up slower than Standard Storage is not.  What’s going on here?  To find this out after we moved our infrastructure to VMs, was a huge disappointment and it slowed down our data pipeline considerably.

Issue #3: Random VM Outages & Useless Support

Sometime after we moved to VMs in April 2015, we began to have these “Critical Kernel Power” outages on our VMs.  The VM would just instantly go dark, without any orderly shutdown.  The event viewer would record this little gem:

“The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.”

In April, we lost both VMs for about 90 minutes and the issue (or at least the same symptoms) recurs every 10 days or so on each VM.  Luckily, it’s only been 1 VM at a time so our services have stayed “up” but this is a low quality of service for 2015.  We provided more details in this post.

Issue #4: Bizarre & Erroneous Billing

When we moved to VMs, we also deployed our web apps on to Azure Web Apps - with everything running in the same data center.  The Azure Web Apps connect to the VMs using an “Azure VNet” - this is just how Azure does it.  

But I noticed on my bill in April that it was about $1k more than I had expected.  Why?  All the traffic between the Web App and the VMs (in the same data center) was being metered and billed!  WTF?  Bandwidth charges in the same data center?

Anyway, endless support and months later, I did receive a credit.  But in the interim, I was covering up to $100/day in, what turned out to be, erroneous charges (nearly $4k in total).  Oh, and for good measure, the month after Azure acknowledged the mistake and credited my account - you guessed it - they charged me again for the same VNet bandwidth.  I’ve still got to get that one sorted with them for the last couple of months.

This kind of thing just saps time and resources - a major headache.

Issue #5:  Cost

In the end, it was the culmination of issues #1-4 (and a bunch of smaller grievances) that sent us looking for alternatives.  But our first motivation wasn’t cost - not by a long shot.  In the end though, we moved to physical hardware with local SSD.  

Our old setup of S2 Web App + 2x D12 VM (w/ P30 Premium Storage Disk) was running about $1150/month (not including the erroneous VNet charges).

The new setup with 2 physical machines, Windows licence, 1TB local SSD in Raid 10, load balancing, and firewall runs about $750/month.  It’s a month-by-month charge (although there was a $130 per machine one-time setup).  And these physical machines have 50% more cores (6 compared to 4) and 225% more RAM (64GB compared to 28GB).

We acknowledge completely that comparing two physical machines to the Azure setup isn’t “apples to apples” - but we will point out, that the two environments service the exact same need.

We’re now 14 days in on the new platform, and a few days ago we moved our last service over.  No issues (to date), way more capacity, and our monthly bill has dropped $400 (about 35%).  

Goodbye Azure - it’ll take a lot to win us back!