Kubernetes Developer at Your Service

[kubernetesdeveloper-blog]

SETTING UP AWS KUBERNETES USING KOPS

Kubernetes is used to maximize the utilization of resources like memory and CPU through open source container management. As a result, it will allow the cluster nodes to have an efficient distribution of applications.

First, you will need to have an AWS account before you can use kops to set up Kubernetes. Some other requirements to get you started includes

· AWS cli setup configured with your secret AWS account key and secret access key

· A DNS hosting Kubernetes for Amazon Web Service

· Kubectl

· S3 Bucket Procedure

Step 1: First install a kops tool

You can follow the procedure to gent kops installed

Go get –dk8s.1o/kops

Cd${GOPATH}/src/k8s.1o/kops/make

After this procedure, you will have kops installed and you will be good to forge ahead.

Step 2: Create Route 53 Domain for the cluster.

Since you are using Distillery.com your domain name should have k8s.distilery.com. The DNS domain name will be easier to remember than the IP  numbered address. You know the IP address mostly employs the use of numbers

Step 3: Creation of S3 buckets

S 3 buckets are used for storage; you can create them online or using cli. It is easier doing this using the cli. You can name you can add Storage as a prefix at the back of your domain name to make things easier.

E.g. Your Domain name is k8s.distillery.com. So your S3 Bucket name could be Storage.k8s.distillery.com

Step 4: Configure your cluster and create clusters

You will start by configuring clusters. This allows kops to output commands to manage your clusters. Then afterwards, you can create your clusters by following the following command

${GOPATH}/bin/kops update cluster –state=${STATE_STORE} ${NAME} –yes

Once you note three nodes listed by the command, then you will be assured that your launch has succeeded. You can now use your AWS kubernetes.

to learn more about AWS and AWS Kubernetes, visit https://distillery.com/

[awsfeed-deactivated20190825]

Top 20 most visited Amazon DynamoDB Developer Guide pages so far in 2018

The Amazon DynamoDB Developer Guide provides an overview of concepts and detailed instructions for using the various features of Amazon DynamoDB. The following list contains the 20 most visited pages in the guide during the first two months of 2018. We’ve included a brief description with each link to explain what each page covers. Use this list to see what other AWS customers have been viewing and perhaps to pique your own interest in a DynamoDB topic you’ve been meaning to learn about.

Working with Queries Learn about the Query operation and how you can use it to find items based on primary key values. You can query any table or secondary index that has a composite primary key (a partition key and a sort key).

Getting Started with DynamoDB These language-specific tutorials help you learn more about DynamoDB. Run the included sample code against either the downloadable version of DynamoDB or the DynamoDB web service.

Setting Up DynamoDB Local (Downloadable Version) You can use the downloadable version of DynamoDB to write and test applications without accessing the DynamoDB web service. When you’re ready to deploy your application in production, just make a few minor changes to the code so that it uses the DynamoDB web service. This downloadable version also helps you save on provisioned throughput, data storage, and data transfer fees.

Best Practices for Tables We recommend that you follow these best practices when working with tables in DynamoDB.

Limits in DynamoDB These are the current limits within DynamoDB (or no limit, in some cases). Each limit applies on a per–AWS Region basis, unless otherwise specified.

DynamoDB Core Components In DynamoDB, you work with tables, items, and attributes. Learn more about these core components.

Working with Items in DynamoDB In DynamoDB, an item is a collection of attributes. Each attribute has a name and a value. DynamoDB provides four operations for creating, reading, updating, and deleting items, and this page describes how to use these operations.

Amazon DynamoDB: How It Works This introductory section provides an overview of DynamoDB service components and how they interact.

Throughput Capacity for Reads and Writes When you create a table or index in DynamoDB, you must specify your capacity requirements for read and write activity. After you define your throughput capacity in advance, DynamoDB can reserve the necessary resources to meet the read and write activity your application requires, while ensuring consistent, low-latency performance.

Node.js and DynamoDB In this tutorial, you use the AWS SDK for JavaScript to write simple programs to perform DynamoDB operations such as creating tables and running queries.

Global Secondary Indexes Some applications might need to perform many kinds of queries, using a variety of different attributes as query criteria. To support these requirements, you can create one or more global secondary indexes and issue Query requests against these indexes.

Working with Tables in DynamoDB Learn how to use the AWS Command Line Interface (AWS CLI) and the AWS SDKs to create, update, and delete tables.

Python and DynamoDB In this tutorial, you use the AWS SDK for Python (Boto 3) to write simple programs to perform DynamoDB operations.

Condition Expressions To manipulate data in a DynamoDB table, you use the PutItem, UpdateItem, and DeleteItem operations. For these data manipulation operations, you can specify a condition expression to determine which items should be modified.

Setting Up DynamoDB (Web Service) Learn how to set up the DynamoDB web service, including all the basic requirements.

Using the CLI You can use the AWS CLI for ad hoc operations, such as creating a table. You also can use the AWS CLI to embed DynamoDB operations within utility scripts.

Naming Rules and Data Types This section of the developer guide describes the DynamoDB naming rules and the various data types that DynamoDB supports.

Improving Data Access with Secondary Indexes DynamoDB provides fast access to items in a table by specifying primary key values. Many applications benefit from having one or more secondary (or alternate) keys available to allow efficient access to data with attributes other than the primary key. Learn how you can create one or more secondary indexes on a table and issue Query or Scan requests against these indexes.

Error Handling Learn more about runtime errors and how to handle them, including error messages and codes that are specific to DynamoDB.

Update Expressions Learn how to use the UpdateItem operation to update an existing item in a DynamoDB table. You provide the key of the item you want to update. You also provide an update expression that indicates the attributes you want to modify and the values you want to assign to them.

Summary

These have been our most popular developer guide pages during the first two months of 2018. In the Comments section below, let us know if you want to see anything expanded or updated on these or other DynamoDB documentation pages to make them more useful to you.

About the Authors

Mike Curtis is a documentation manager for Amazon Web Services.

    Israel Santos is a senior programmer writer for Amazon DynamoDB and Amazon SimpleDB. He works with other writers and editors to provide and improve documentation for AWS services.

[awsfeed-deactivated20190825]

‘The Cost of Doing Business’ - Think FaaS Podcast

Listen to “Think FaaS with Trek10”

Transcript

Hi, I’m Forrest Brazeal at Trek10, and this is ‘Think FaaS’, where we learn about the world of serverless computing in less time than it takes to run a Lambda function. So put five minutes on the clock - it’s time to ‘Think FaaS’.

Today I want to talk about the cost of doing business with serverless. I use the word “business” deliberately because, as much as we engineers tend to lose sight of the fact, technology switches don’t happen just because it’s fun to play with new stuff. If you want to go serverless, you have to show why it helps your business win. And one way we can answer that question, not the only way, is by looking at cost. I’ve got three ways that you can put the cost of serverless to work for you.

First, embrace functional billing.

This is the idea that you’re only paying for resources you actually consume, not for provisioned capacity that may end up just being wasted overhead. AWS Lambda is the canonical example of this concept. When you call a Lambda function to handle an API request, it maybe runs for a hundred milliseconds, and you get billed for a hundred milliseconds. And a hundred milliseconds of cloud compute is really, really cheap. For Lambda, even once you get past the free tier which is a million requests per month, you’re looking at something like a millionth of a cent per call. Even at hundreds of thousands of requests a day, your compute costs are going to look like a rounding error - a few dollars a month. We show these calculations to folks that are used to traditional server overhead costs and they just start giggling. It’s almost hard to believe.

This is why you hear stories about people saving 90 to 95% of hosting costs by switching to serverless. It’s not because serverless is magic, it’s just that those traditional systems were so massively overprovisioned for the amount of work they were actually doing. On the other hand, with Lambda, it’s actually a lot easier to max out the default service limits of your AWS account than to exceed your budget.

Second, you have to leverage the power of services.

If you make a spreadsheet and put the hourly cost of your serverless infrastructure - your Lambda functions, your Dynamo tables, and what have you - on one side, and the cost of your traditional on-premise or cloud VM-based infrastructure on the other side, you may very well see a phenomenal savings in favor of serverless, but you also won’t see the whole story. The cost savings of serverless extend way beyond the fact that you’re not paying for servers.

Let’s take logging as an example. Let’s say you’re looking at a managed logging service like Splunk or SumoLogic. And you can see up front that it’s going to cost you ten thousand dollars a year to pay for the licenses you need and the amount of data you have. Now you say, “Well, if I build my own logging stack with ElasticSearch, LogStash, and Kibana, and I run it on ECS in AWS, and I store the data in S3, I would only spend eight thousand dollars in hosting costs.”

Great. But how long does it take your engineers to build that system? And how much are you paying them per hour? And how much will you continue to pay them to maintain it over time, especially given that they’re not experts in building a system like that and will run into lots of bugs? Unless you’re so huge that there really is economy of scale in hiring somebody to solve those edge cases and build infrastructure tooling full time, the smart play would be to take your $10k and sign up for the managed service, because your logging stack doesn’t differentiate you. Nobody’s buying whatever your product is because you personally designed a beautiful logging stack behind the scenes. Let your developers focus on the business-critical stuff.

Finally, strive for increased efficiency, not reduced spend.

I don’t want to give you the idea that a serverless system is necessarily cheap in absolute terms. Compute may be a rounding error, but we’ve found here at Trek10 that as you scale up a serverless system, bandwidth and storage costs will definitely not be trivial. Now, the important thing to note is that those are both usage-based metrics. If you’re not actively transferring or storing data, you’re not incurring as much cost. And all of a sudden we’re tying the cost of our system very effectively to its success.

There’s a guy named Simon Wardley - who’s kind of a prophet when it comes to cloud strategy, and I definitely recommend checking him out - who I talked to for my Serverless Superheroes series at A Cloud Guru. He is very strong on the idea that serverless is not going to reduce IT budgets. He points out that you can look at history and see that anytime humans make something cheaper and easier to access - in this case, cloud computing - we consume more of it rather than less. I like to say that “spend expands to fill the available space.” But what you do reduce, hopefully, is the amount of waste in your budget. With serverless, your spend reflects your actual usage of resources, and ideally that number is tied more closely to your bottom line.

Thanks for listening! If you have a cool story about how serverless is helping your business win, I would love to hear from you on twitter @forrestbrazeal. You can also follow Trek10 there @Trek10inc. And I’ll catch you on the next episode of ‘Think FaaS’.

[awsfeed-deactivated20190825]

Artificial Intelligence as a Service: AI Meets the Cloud

In their never-ending quest to stay ahead of the competition, a growing number of enterprises are looking for ways to incorporate artificial intelligence into their applications, products, services and big data analytics. And one of the easiest and most popular ways to get started with the technology is by using cloud-based AI as a service (AIaaS) offerings.

According to IDC, worldwide spending on cognitive and AI systems is expected to increase at a compound annual growth rate (CAGR) of 50.1 percent through 2021. That means total spending on these technologies will increase from $12.0 billion in 2017 to a whopping $57.6 billion by 2021.

Much of that spending will likely go to cloud-based AI services. The RightScale 2018 State of the Cloud Survey revealed that organizations are particularly interested in using cloud services related to one particular type of AI: machine learning. When survey respondents were asked which types of public cloud services they planned to use in the future, machine learning services were the number one vote-getter, with 46 percent either experimenting with the technology or planning to deploy it, despite the fact that only 12 percent are currently using those services.

Clearly, organizations are interested in AI as a service, and the cloud vendors are responding with a growing number of products.

Types of AI as a Service

There are myriads types of AI as a Service because “artificial intelligence” is a broad term that covers a wide array of technologies. At its core, AI is about creating machines that can do the same sorts of things that human brains can do. For example, AI includes computer vision technologies that can see and identify the objects in pictures. It also includes natural language processing technologies that enable systems to carry on a normal conversation, as well as machine learning technologies that allow computers to learn without being explicitly programmed.

AI as a service offerings make one or more of these types of artificial intelligence technologies available as a cloud service. Currently, the AI as a service products on the market generally fall into the following categories:

Bots and digital assistants:For many people, the first thing that comes to mind when they hear the phrase “artificial intelligence” is a digital assistant like Apple’s Siri, Microsoft’s Cortana or Amazon’s Alexa. These tools use natural language processing technology to carry on conversations with users, and many also use machine learning to improve their skills over time. Many enterprises want to add similar functionality to their products and websites. In fact, according to IDC, the AI use case that saw the most spending in 2017 was automated customer service agents. But creating your own bot from scratch is a monumental undertaking. As an alternative, several vendors offer bot platforms as a service. Organizations train the bots with their own data and then use them to answer simple questions, freeing up human customer service agents for more complicated tasks.

Cognitive computing APIs:An application programming interface (API) makes it easy for developers to incorporate a technology or service into the application or products they are building. The leading cloud vendors all offer an assortment of APIs for that allow developers to add a particular type of AI to their applications. For example, a developer that wants to make a photo-sharing app might use a facial recognition API to give the app the ability to identify individuals in pictures. Thanks to the API, the developer doesn’t have to write the facial recognition code from scratch or even thoroughly understand how it works. He or she uses the API to allow the app to access that functionality in the cloud. APIs are available for a wide variety of different purposes, including computer vision, computer speech, natural language processing, search, knowledge mapping, translation and emotion detection.

Machine learning frameworks: These tools allow developers to create applications that can improve over time. Generally, they require developers or data scientists to build a model and then train that model using existing data. Machine learning frameworks are particularly popular in applications related to big data analytics, but they can be used to create many other types of applications as well. Accessing these frameworks in the cloud can be easier and less expensive than setting up your own hardware and software for machine learning tasks.

Fully managed machine learning services: Sometimes organizations want to add machine learning capabilities to an application, but their developers or data scientists lack some of the skills or experience necessary. Fully managed machine learning services use templates, pre-built models and/or drag-and-drop development tools to simplify and expedite the process of using a machine learning framework.

The “holy grail” of AI as a service would be to create a general artificial intelligence that could be accessed as a cloud service. A general artificial intelligence is a computer system that can think and communicate in all the same ways that humans can. Most experts believe that researchers are still many years away from creating general AI, if they will ever be able to do so at all.

Benefits of AI as a Service

Some organizations, mostly very large enterprises, choose to invest in their own AI research and hardware. However, many prefer to use AI as a service because this approach offers a number of benefits, including the following:

Advanced infrastructure: AI applications, particularly machine learning and deep learning applications, perform best on servers with multiple, very fast graphics processing units (GPUs) that run workloads in parallel. However, those systems are very expensive, putting them out of reach for many organizations and use cases. AI as a service gives organizations access to those superfast computers at a price they can afford.

Low costs: Not only does AI as a service eliminate the need to pay for expensive hardware upfront, it also allows organizations to pay only for the time that they need that hardware. In cloud computing jargon, most AI workloads are said to be “bursty,” that is, they require a whole lot of computing power for a short period of time. AI as a service charges organizations only for what they use, lowering their costs significantly.

Scalability: Like other types of cloud services, AI as a service makes it very easy to scale. Often organizations start with a pilot project that allows them to see how AI could be useful. With AI as a service, they can quickly move that pilot project into full production and scale up as demand grows.

Usability: Some of the most best artificial intelligence tools are available with open source licenses, but while they are inexpensive, these open source AI tools aren’t always very easy to use. The cloud AI services generally make it easier for developers to access artificial intelligence capabilities without requiring them to be experts in the technology.

Drawbacks of AI as a Service

The two biggest drawbacks of AI as a service are two issues that are common to all cloud computing services: security and compliance.

Many AI applications — especially applications that incorporate machine learning capabilities — rely on vast quantities of data. If that data is going to reside in the cloud or be transferred to the cloud, organizations need to make sure that they have in place adequate security measures, including encryption both at rest and in transit.

In some situations, regulations may prevent some types of sensitive data from certain industries from being stored in the cloud. Other laws require that some data remains within the borders of the country where it was originated. In these cases, it may not be possible to use an AI as a service offerings for those specific use cases.

Another potential drawback is that AI as a service can be very complex. Organizations will have to invest time and effort in training and/or hiring staff with artificial intelligence and cloud computing skills. However, many organizations believe that this hurdle can be easily overcome and that AI as a service will pay off in the long run.

AI as a Service Vendors

All of the leading cloud computing vendors offer AI as a service, and some smaller vendors has cloud-based AI services as well. Here’s an overview:

Other AI as a Service Vendors

The list of additional entrants into the AI as a Service sector will certainly grow rapidly. As a start:

Oracle

Salesforce myEinstein

[indeed-very-far-away-deactivate]

like “how am I going to support a highly available application on AWS that can scale to meet arbitrary load but remains easy to update and redeploy?” The answer? Use Kubernetes. And if that don’t work, use more Kubernetes.

*sound of Scout dying off in the distance*

[dataandme]

Microservices with Containers

The microservices architecture is far from a new trend; it’s generally accepted as a better way to build apps these days…

“Basically, the microservices approach in a nutshell dictates that instead of having one giant code base that all developers touch, that often times becomes perilous to manage, that there are numerous smaller code bases managed by small and agile teams.”