#horizontal scaling

[techman1010]

Vertical Scaling and Horizontal Scaling: Benefits and Differences

As businesses grow and their demand for computing resources increases, ensuring their IT infrastructure can scale effectively becomes crucial. Two common approaches to scaling systems are vertical scaling and horizontal scaling. Both have their unique benefits and are suited for different types of workloads and growth patterns. Understanding the differences and the advantages of each can help businesses make informed decisions about optimizing their infrastructure.

Vertical Scaling (Scaling Up)

Vertical scaling, also known as scaling up, involves increasing the capacity of a single machine by adding more resources, such as CPU, RAM, or storage. Essentially, you are making one server more powerful to handle larger workloads.

Benefits of Vertical Scaling

Simplicity in Management Vertical scaling is easier to manage since there is only one machine or a small number of machines to configure and maintain. This reduces complexity and simplifies monitoring, updates, and maintenance tasks. As all data and processes remain on one server, managing and troubleshooting are straightforward.

Faster Implementation Upgrading an existing server with more resources like memory or processing power can be done relatively quickly. You don't need to redesign or redistribute applications across multiple servers, making vertical scaling a faster option when an immediate performance boost is required.

No Changes in Application Architecture In vertical scaling, the underlying application architecture remains unchanged. Since everything stays on a single machine, developers and system administrators do not need to make major changes to the application to handle more traffic or processing power. This makes it easier to upgrade without disrupting operations.

Cost-Efficient for Small to Medium Workloads For smaller applications or workloads, vertical scaling can be more cost-effective. Instead of investing in multiple machines, companies can maximize the potential of a single, powerful server to meet their needs.

Horizontal Scaling (Scaling Out)

Horizontal scaling, or scaling out, involves adding more machines (servers or nodes) to a system to distribute the load. Rather than making one machine more powerful, horizontal scaling allows businesses to spread workloads across multiple systems, which can be added as needed.

Benefits of Horizontal Scaling

Increased Fault Tolerance One of the major benefits of horizontal scaling is its fault tolerance. Since workloads are distributed across multiple servers, the failure of a single machine is less likely to disrupt the entire system. The system can continue to function with minimal downtime, as other servers pick up the slack.

Unlimited Scalability Unlike vertical scaling, which is limited by the physical constraints of a single machine, horizontal scaling offers virtually unlimited scalability. New machines can be added as demand grows, allowing businesses to scale infinitely in response to increasing workloads.

Cost-Effective for Large-Scale Operations For large-scale applications or enterprises handling significant traffic, horizontal scaling is often more cost-effective. Rather than investing in a high-cost, ultra-powerful server, companies can deploy several low-cost servers to handle workloads in parallel. Additionally, with cloud solutions, it’s possible to pay only for the resources used.

Improved Performance with Load Balancing Horizontal scaling often comes with load balancing mechanisms that ensure traffic is evenly distributed across multiple machines. This leads to improved performance, as no single server is overloaded. Applications can handle more users, traffic, or data processing tasks without bottlenecks.

Better Adaptation to Cloud Environments Horizontal scaling fits seamlessly with cloud environments and microservices architectures. Cloud platforms like AWS, Azure, or Google Cloud offer services that allow easy scaling out by provisioning new virtual machines on demand. This ensures businesses can scale quickly and dynamically according to real-time needs.

Vertical Scaling vs. Horizontal Scaling: Key Differences

Scalability Limitations Vertical scaling is limited by the maximum capacity of a single machine. Once the server reaches its physical limitations, you can no longer scale up. Horizontal scaling, on the other hand, allows you to keep adding machines, making it more suitable for long-term scalability.

Cost Structure Vertical scaling often involves upgrading to more powerful (and more expensive) hardware. This can lead to high upfront costs. Horizontal scaling distributes costs more evenly, allowing companies to pay for additional servers as they grow.

Performance While vertical scaling can deliver immediate performance improvements, it may become inefficient for handling large-scale, distributed workloads. Horizontal scaling distributes the load, improving performance without over-relying on a single machine.

Application Architecture Vertical scaling usually does not require significant changes to application architecture, while horizontal scaling often requires a more distributed approach. Applications must be designed to run across multiple servers or nodes in a horizontally scaled environment.

Choosing the Right Approach

When deciding between vertical and horizontal scaling, consider the size and growth potential of your business, as well as your application's architecture. For small-scale applications or those that don’t require frequent scaling, vertical scaling might be more practical. However, for businesses expecting rapid growth or handling large amounts of traffic, horizontal scaling provides a more scalable, resilient, and cost-effective solution.

Conclusion

Both vertical and horizontal scaling have their advantages depending on the specific needs of your business. Vertical scaling offers simplicity and quick upgrades, while horizontal scaling provides better long-term scalability and fault tolerance. The key is to assess your current and future needs, ensuring that your infrastructure can grow alongside your business.

[techbrace]

Vertical Scaling and Horizontal Scaling in Cloud: The Difference

The most striking factor about cloud hosting is its scalability. Scalability is the ability to modify or expand the configuration inorder to  contain the load. This article gives you an insight on how the scaling is done in cloud hosting. Read more from  https://medium.com/@stephinsteev/vertical-scaling-and-horizontal-scaling-in-cloud-the-difference-67b734518667

[johncloud]

Cloud Orchestration Platform eNlight 360°

Management of a cloud infrastructure becomes a tiresome challenge for in-house IT staff when massive IaaS and PaaS deployments take place on the cloud. Using programming to manage the interconnectivity and communication between workloads on public cloud and private cloud infrastructure is known as Cloud Orchestration.

Cloud Orchestration solutions are designed to create a cohesive workflow of automated tasks and achieve goals with strict supervision and policy enforcement; this serves as a catalyst in propelling your infrastructure into the next level of readiness for advancing digitisation of the business world.

Cloud orchestration is a single centralised approach integrated to handle all the data, software, assets and networks, thus aiding in automating complex business problems such as costing processes or policy enforcement, reducing the need for human monitoring and supervision. This results in lowering the costs as well as the chance for errors.

eNlight 360? orchestration software offers services with robust features like automatic vertical scaling for resource provisioning on demand, as well the conventional horizontal scaling based on software-defined load balancing.

Let us look at these features and some additional ones in detail to get the complete idea of how eNlight 360? Cloud Orchestration can help your organisations in optimising multiple workflows, reducing friction among various teams and in general enhance your business performance.

Read More>>

[timetocode]

A diagram of an infinite mmo world made with nengi.js instances. I’ll follow this up with a technical post that describes how it all works and in what challenges it might experience as it tries to handle thousands or hundreds of thousands of players.

[nandafirdaus]

Thesis Finished

Today I have finished my thesis. Woohoo!!!

Well, I have to say that it’s not that hard to do your thesis. Well, maybe because I got an easy topic so I didn’t get a lot of trouble to do it.

I took a topic in database, which is a topic for computer science major. My research is about comparison of vertical and horizontal scaling. I measure their performance and analyze which one has a better performance. In this research I use PostgreSQL as the DBMS. Well, I’m very grateful that I didn’t take the IS topic because I have to do an interview and write thicker report if I take the IS topic. My friend Gaby wrote almost 500 pages of report and lely wrote about 350 pages of report. I think that is scary for my printer, budget, and for the environment. :p

If you want know how thick are their, just compare it with my 77 pages thesis report:

So, now I still waiting for the presentation. The secretariat will inform me about the date of the presentation. Wish me luck. :)

[unscramblegk]

Scalable virtual appliances and resource usage

[author:alex]

A common problem with merging a SaaS application's services into one virtual appliance is resource usage.

If your application requires a queue system, an SQL database, some background workers, and various other processes, it might be difficult to run it all in a tiny virtual machine.

One approach is to define a set of Minimum Requirements. This ensures the people using your virtual appliance are allocating sufficient RAM, CPU and DISK. The term for this is vertical scalability. Having the ability to assign a machine's hardware resources however you want is a great benefit of virtualization. Unfortunately, scaling vertically will eventually reach a certain cost/performance limit.

Unfortunately, scaling vertically will eventually reach a certain cost/performance limit.

If your customers are forced to only scale vertically, then they will end-up performing crazy workarounds, and that's never a good thing.

What workarounds?

If you look at GitHub Enterprise, it doesn't provide the ability to define an external database, or to share repositories between two virtual appliances. If a company has 500 concurrent users (lots of developers), they might be required to run multiple GitHub Enterprise instances to support the load on multiple physical servers. Since it doesn't natively support this, the admin would need to allocate repositories in different instances, and maybe even DUPLICATE repositories on each instance, along with using custom git remotes on each developer's workstation, pointing to each instance. Sounds complicated?

Otherwise you'll need to have one incredibly beefy server to handle everything, with expensive/fast disks, multiple CPU cores and maybe even bonded network interfaces to handle excess network throughput.

A nice solution

To alleviate this problem, being limited by one server's physical/vertical scalability, we often suggest our customers to allow for horizontal scalability. This entails two things:

Allowing your customers to change the IP/Networking information of certain services, in order to offload tasks to other servers.

Providing additional virtual appliances to perform those tasks.

Why limit yourself to just ONE virtual appliance? Nobody forced that upon you. If you feel your customer needs two appliances to get maximum performance for your on-premises SaaS application, then give them two appliances!

How do you do it?

The idea is simple: imagine you were your customer, and give them the ability to scale the same way you scale your hosted SaaS application.

Providing your customer with more freedom, and the ability to outgrow one simple virtual appliance has many benefits. It translates to more sales since you can support your customer's growing business.

If you expect your customers to outgrow a virtual appliance and require more resources (across multiple physical servers), then it's a great idea to give them the ability to change certain settings, give them a few options to easily scale out their private virtual appliance infrastructure.

Conclusion

As usual, feel free to email us if you have any questions. Stop by our chat #Jidoteki on Freenode IRC, or send me a message on Twitter. We'll gladly answer your questions.

I also encourage you to check out Jidoteki if you're interested in selling your SaaS, behind the firewall :)

[timetocode]

Horizontal Scaling Brainstorm

One of my goals for a long time has been to experiment with splitting up a game world across multiple cores and multiple physical machines. As I trickle in some legitimate gameplay I can't help but to start imagining how to scale these things. In fact my thoughts were becoming so persistent that I simply had to write them down.

The world can be saved in one place, but hosted on multiple servers

What is this world anyways? Well it is a huge terrain of tiles, divided in chunks, and populated by trees, NPCs, and players. At some point it may have houses, cities, caves, dungeons, and a super annoying parallel dimension with every NPC you've ever killed called ghostland (uh huh). Some of these spaces, such as the terrain of the surface, are continuous, or at least very large. Other spaces, such as a hypothetical cave or dungeon may be finite and small.

From a technical standpoint, the world above could easily be  stored on a single computer. Storing the world and its denizens does not call for horizontal scaling (though if it did, it could be a sharded mongodb).

The subjects that consume the most computing thus far are AI, collisions, and sending network messages. How can these be scaled? One idea is to think of an individual server process as being responsible for a region of a larger world. Perhaps one core is responsible for handling all player movement, collisions, AI and network requests for the area from tile 0, 0 to 5000, 5000. When each server (or process) starts up, it begins by fetching the geography and population of this finite area from storage. A neighboring area, say from 0, 5000 to 5000, 10000, could live on a different process, and so on. Each of these process would answer on a different port, and a player traveling across the boundary of one region to another would be transferred from one process to another. This approach can extend beyond individual cores on a system and could even send a player from one physical server to another.

Artificial intelligence could be its own process, or even weirder

That, if it works, is a great start. But as I develop this prototype-turned-tech-demo, I get the feeling that the AI is going to eclipse the whole quadtree+network component in terms of cpu greed. The very first AI I wrote for this project ran from the client, and sent inputs as if it were a player. In effect it was a bot, not even an in-game bot, more so a hacker's 3rd-party-style bot. This brings up the point that AI can run around in the game world without even running on the server. Now I'm not suggesting that AI should be run on the client, but I am open to the idea of AI running as a separate process on the server, and connecting to the server using sockets just the way a player does.

This AI would be granted quite a bit more data but this usage is largely negated because this is a connection between two processes on the same computer. Another method of accomplishing a very similar outcome is to share state via redis/mongodb/etc, to which I'm also open. Perhaps one AI process exists for multiple regions, or perhaps each region needs is own. Perhaps the AI is so computationally expensive that a physical sever consists of one geography+player process, and seven AI processes. Either way the AI and and the main server process for any given region should be running on the same physical machine so that they can exchange data without consuming any outgoing bandwidth. Perhaps, in a pinch, AI could run on a different machine in the same data center.

The server boundary

In this scenario of a virtual world segmented and hosted across numerous machines there will come a time when a player is standing next to the server line. The player will be standing just inside of the region belonging to server A, but half of the player's screen will be in a region managed by server B. How would this work?

My preliminary thinking is that as a player nears a sever boundary, the server begins copying the player's messages and sending them to the other server(s) responsible for the neighboring region(s). This will create a buffer region around within which players from either server are visible and multiple servers are simulating their actions. The player will be transferred from one server to another as they cross the actual boundary, but both servers are aware of the player before this point, thus keeping it seamless.

If the simulations are deterministic, there's no reason to say that one server has to be authoritative over another server regarding the player's message. If the simulations are not going to be identical, then one server must take charge and tell the other server about the results of a player's actions near the border.

AI entities could similarly cross boundaries, by transferring their little digital souls to the respective server. The servers are running the same code, their only difference is the region which they are responsible for simulating, so this wouldn't produce any differences in the AI's behavior near or after crossing borders.

Minimizing the increased load of server boundaries

Locating a population center right at a server boundary should be avoided. Because a buffer region surrounding the server boundary in effect exists on both servers, at least twice the work is being done in this area. Population centers should ideally find themselves the sole responsibility of one server.

How many servers meet at the server line? Well conventionally speaking, if the server is cut up into squares analogous to my chunking and quadtree systems, then most boundaries would be between two servers but there would be areas where four corners met, too.  A creative allocation of server regions could hypothetically remove the four corners scenario entirely. The following image shows a few ideas:

aligned squares: max. 4 possible servers involved

offset squares: max. 3 possible servers involved

outer region + inner regions: 2 servers involved, region size allocated based on anticipated usage

I'm sure there are some advantages and drawbacks to each method.

As the complexity of the divisions increases, it becomes clear that there is a process somewhere on the server whose job would be to allocate regions and spawn new server instances. This isn't intensive by any means, but it is a job that would be external to the concerns of any particular server instance.

Dynamic horizontal scaling

I can't help but assume that the ultimate destination of this line of reasoning is some form of master server that dynamically allocates and spawns child processes based on need. More than that, such a master server would control more than cores, it would control numerous physical machines, or could tie into a cloud service and requisite resources appropriate to the current load. While I'm itching to design such a thing, I'm still nevertheless going to keep my initial horizontal scaling in a tidy unchanging grid far away from any cloud.

Small regions: scaling down and granularity

Tiny isolated areas won't need a whole server much-less multiple servers. These types of areas will be maintained in groups on a single process. Finite caves, dungeons, and perhaps even the interiors of a house (if it is the type of game where an interior is a separate zone) qualify as small regions that need to be piled together and simulated together. The AI for these regions will be treated in an analogous way.

This is a change in definition: a server process has one or more regions for which it handles all players, network data, and maybe NPCs. This architecture adds granularity, and also presents the option of scaling down. Multiple regions constituting a larger whole can be hosted on one server as they empty out, while smaller regions that found themselves distributed across multiple machines during peak usage would all end up in a single process. Or work could be better distributed: a process holding nothing but a giant but sparsely populated forest could then be asked to host a section of an overpopulated city in addition.

I can't wait to experiment with this stuff.

[code-snippets-tech-talk]

Auto Scaling - SLA, Throughput and Queueing

SLA (Service Level Agreement) for a computing service is usually defined in the following language - X% of requests will be handled within Y units of time. For example, an ad server might have the following SLA - 99.9% of requests will be handled within 500ms, 99% of requests will be handled within 200 ms.  Typical SLA definitions include a ceiling on calculated values like tp99. tp99 means the minimum time it took to process 99% of the requests handled by the service in a time period. Generally, times used in SLAs don't include client latency (ie latency from client location to datacenter). However, CDN (Content Delivery Network) like Akamai, CloudFront SLA would include the client latency as well as those services are designed specifically for that purpose.

For a service, there are 3 important factors to monitor

Capacity - Number of homogeneous servers providing the service

Throughput - Units of work done in a fixed amount of time.

Processing Time - Time taken from receiving request to providing response.

Auto Scaling goal is to use minimum capacity and find optimal load balancer configuration while keeping the processing time under SLA. In a typical auto scaling group, the homogeneous servers will be behind a load balancer. The load balancer is assumed to provide the following features - weighted distribution, maximum number of concurrent jobs on each server, request queue when servers have reached maximum number of jobs. Popular load balancers like F5, haproxy provide these features.

In a typical server, the relationship between number of concurrent requests and processing time/throughput looks like the graph below. We want to find the maximum throughput by finding the optimal value for number of concurrent requests getting processed on server while still being under SLA limits. I will be referring to the optimal number of concurrent requests as maxconn.

Maxconn(server) = value learned over time by Auto Scaler to achieve maximum throughput while server processing time is under SLA

The Auto Scaler can use load balancer logs as a central point of collecting all the metrics since both requests and responses are passing through that system (unless using Direct Server Return). At regular intervals, Auto Scaler will collect current processing time and throughput of each server. Using that data, we will calculate the following and feed them back into load balancer.

Weight(server) = 1 + FLOOR(100*MAX(0,1- Current server processing time / max processing time as per SLA))

For example, SLA defines that tp95<=500ms. Let's say current tp95 for that server is 405 ms. Weight of server = 1 + FLOOR(100*MAX(0,1-405/500)) = 20

The server weight computed here is used by load balancer to decide load distribution for incoming requests. This allows the Auto Scaler to adjust weight on the fly and move traffic dynamically to lesser loaded servers. For a situation where most jobs are comparable, this dynamic rebalancing might not look that interesting. However, most services have requests which vary in size. Some requests are compute intensive, while some  are just serving some static data which takes little server resources. The dynamic load balancing is one of many ways to model remaining server capacity and distribute load to servers under light load. Another way to model dynamically load balance can be based on CPU usage where server weight depends on average CPU usage on that server.

While the above handles the load distribution, we will use the queuing feature of load balancer to park the requests while the servers are busy. When all servers reach maxconn, the load balancer will start queuing requests. Note that queuing is an important feature of load balancer and can provide great results when used properly. Like we see in the graph above between number of concurrent requests and throughput, giving servers more work than what they can handle in parallel reduce throughput. So, we can achieve better service throughput with a queue than without one.

Service processing Time = Time spent in queue + Server processing time

Since, time spent in queue is also adding to service processing time, we need to ensure that queue length is not making the service break SLA limits. So, we will periodically check that Processing Time of service <= Max Processing Time of service as defined in SLA. For Auto scaling purposes, we will define a monitor, let's say

Auto Scaling Monitor = Processing Time of Service / Max Processing Time of Service per SLA.

When Auto Scaling Monitor reaches a certain high ceiling (say 0.9), we will add more capacity to the service so that Auto Scaling Monitor goes below threshold. So,

Number of New Servers = CEILING((Auto Scaling Monitor value - defined monitor ceiling) * Current Capacity)

Similarly, when the Auto Scaling Factor reaches a certain floor, we will reduce capacity on the service so that servers stays utilized

Number of Servers reduced = FLOOR((Auto Scaling Monitor value - defined monitor floor) * Current Capacity)

There are some additional checks required while reducing capacity, such as ensuring that reducing capacity will not make Auto Scaling Monitor touch ceiling (to avoid a continuous scale up and scale down) and there is at least enough capacity to handle 1 server failure etc.

Using the ideas mentioned above, Auto Scaler can add and reduce capacity on demand while ensuring that the service is performing under SLA limits.