#web-service

[ichiroku11]

国産のツールでは初じゃないでしょうか。 Webサイトやスマホアプリのページデザインを積み木感覚で楽しくデザインできる無料のオンラインサービスを紹介します。 操作が非常に快適で、ちょっと触るだけでもかな

[ichiroku11]

今年になって人工知能、AIが取り入れられたWeb制作関連のサービスがどんどんリリースされています。フォントの組み合わせをAIで、カラーの組み合わせをAIで、サイトのリデザインをAIで、自動化することが

[gokulvanan]

Batch Service calls

This post is to share my experience in building complex Apps over third-party web services. The post focuses on design challenges when building batch-processing services,

  We all know that when measuring performance there are two things to consider.

One is the cost of operation and the other the frequency of operations. 

  for ( Object o : objLst ){ // freq of operation

  o.doSomething();      

  Response resp = makeWebServiceCall(o);  // cost of operation

  processResponse(resp,o)

}

  Where the cost of operation is usually not considered by a programmer.

We focus only on reducing the frequency of operation using Algorithms and Data structures.  This works fine cause usually the cost of operation usually depends on the underlying hardware and can only be improved with better hardware. But when it comes to building Web services, which call other third-party services in synchronous fashion the cost of operation is very high, as latency cost could grow based on the location of the third-party service server boxes.

  Hence comes the need for batch calls.  The role of batch calls is when making POST/PUT call to the REST service to create/update/delete.  The idea is to avoid making N independent calls for N independent entities and batch them up.

  This involves creating services that build Array of entities and send all these to the third-party service in one shot.  Now this requires a little different approach to the problem.

  This involves more iterations to preprocess the data make single web service call and post process the data and also requires proper error handling at every stage.

Hence there would more loops to do each of these preprocessing steps but this would still be very efficient then making N web-service calls.

Error Handling becomes tricky when operations are transactional. Consider the case when:

 Your single action needs to

1)   Create List Entity A objects.

2)   For each Entity A create List of  Entity B and One Entity C

Here there are two approaches depending on the kind of third-party web service.

If the third-party web service provider is doing things right you may have batch service call capable of taking Objects with subObjects like:

   [ { name: “Entity A”, listofB: [ { name:”B1”},…] ,  C :{ val: “this is C”}}, … ]

  But most cases this may not be supported. You may be required to create Entity C then Entity B  and pass their references while creating Entity A.

  Here is where Error handling becomes Tricky. Depending on Atomicity of transaction. Consider the case

Entity C is created.

Enity B1, B2, B5 are created but B3, B4 where not because of some error

  At this stage there are two approaches to handling error if the Transaction has to be Atomic. (i.e. All good or Rollback)

1)   Retry approach - Remake all failed webservice calls again (Works better if the third-party web services are idempotent as then we can remake all webservice calls)

2)   Rollback approach – Rollback – by making calls to delete the created Entites.

Note: Both approaches above require retry thresholds to be set and the failsafe code of logging into the file system when all successive attempts fail.  A techops monitoring system similar to Nagios would be needed to monitor these logs and alter such failures.

  The above approaches sometimes could make things slow. But it all depends on the reliability and availability of the third-party service provider.  The RollBack Approach is what I have done while working with Facebook API’s  at work.

  But even better responsiveness can be achieved by taking an Asynchronous approach to this problem.

  Note: By Async I don’t mean NonBlocking server approach [

i.e. working with NonBlocking embedded sever (Like Netty) rather than external containers like Tomcat. (Correction I believe the latest version of Tomcat has async components built in). Its obvious that nonbolcking servers are the better alternative to threaded servers in terms of load handling but the logic that goes behind serving the request plays the role in responsiveness.]

  There are two asycn approaches:

  1)   Creating Parallel execution context with synchronous response to request

2)   Immediate response to request with background processing.

  The first approach involves using parallel executions through different thread execution context, which solves the processing logic faster.

  The second approach involves persisting the request state and responding immediately with processing notification. The actual processing happens in the background either through cron based scheduler or Producer – Consumer trigger mechanism. 

[cuncode]

Some APIs for you

It's no secret that I am a huge fan of Web APIs. Working with Web APIs is full of fun. Below is some of APIs for you:

VNTextPro

NPL API for Vietnamese: word segmentation, sentence segmentation, post tag. Support 2 engines: vnTagger & JVnTextPro.

Hosted by Heroku

http://vntextpro.api.olragon.com

OpenOCR

OpenOCR is open source OCR-as-a-Service using Tesseract and Docker.

http://openocr.api.olragon.com

PortScanner

PortScanner is PortScanner-as-a-Service written in node.js. Blog post (Vietnamese) http://vietjs.com/node-js/quet-cong-mang-sieu-nhanh-su-dung-node-js/

http://portscanner.api.olragon.com

[mithunmo]

RESTFUL Webservices

A nice article on web services

http://www.peej.co.uk/articles/restfully-delicious.html

[cknbstr-blog]

[cknbstr-blog]