Sunday, September 22, 2019

RxJava


Learning RxJava: a good book published in 2017


Cold Observables: pull; replay the emissions to each Observer; most likely for finite dataset.

Hot Observables: broadcast the same emission to all observers at the same time; most likely for infinite dataset.

ConnectableObservable is hot.  publish() or replay() will convert any observable to hot observable. connect() will fire emissions.

Using ConnectableObservable to force each emission to go to all Observers simultaneously is known as multicasting

Subject: is both Observable and Observer. acting as a proxy mulitcasting device (kind of like an event bus).  It is "mutable variables of reactive programming"

The simplest Subject type is the PublishSubject, which, like all Subjects, hotly broadcasts to its downstream Observers.

Subjects act like magical devices that can bridge imperative programming with reactive programming.

you will most likely use Subjects for infinite, event-driven (that is, user action-driven) Observables.

the onSubscribe(), onNext(), onError(), and onComplete() calls are not thread safe!

Following will safely sequentialize concurrent event calls so no train wrecks occur downstream:
            Subject<String> subject =
                    PublishSubject.<String>create().toSerialized();

Use subscribeOn(Schedulers.computation()) to use a thread pool to emit concurrently.
use observeOn() to intercept each emission and push them forward on a different Scheduler

when you have a flow of 10,000 emissions or more, you will definitely want to use a Flowable (which supports backpressure) instead of an Observable.

.buffer()
.window() for batch
.sample(, TimeUnit.MILLISECONDS) for sampling
.throttleFirst()
.onBackpressureBuffer(16, () -> {}, BackpressureOverflow.ON_OVERFLOW_DROP_OLDEST) for customized action when the buffer fills up
.onBackpressureDrop for dropping


Sunday, September 15, 2019

Spring & Guice DI


Spring


The way to use @Autowired and register beans:


 @Component simple bean
 @Controller servlet control ben
 @Repository DAO classes
 @Configuration general bean
 XML file define bean from xml 

Also use @Bean to register a bean on method level.

@ComponentScan to detects and instantiates annotated components automatically




Guice


@Inject instead of @Autowired

@Provides instead of @Bean

Governator plugin instead of @ComponentScan


Guice Injection Styles
LOCATION
INJECTION ORDER
MOTIVATION
COMMENT
Constructor
First
Class immutability1 Mandatory dependencies
Only one allowed with @Inject.
Field
Second
Quick prototyping Code that doesn’t need testing
Injection order is random.
Setter
Third
Dealing with legacy classes
Optional dependencies
2
Injection


More info:

https://www.baeldung.com/guice-spring-dependency-injection
https://www.amazon.com/Google-Guice-Lightweight-Dependency-Firstpress/dp/1590599977


Monday, September 2, 2019

Kafka, Spark, and Elasticsearch (architecture comparison)





Kafka Spark Elasticsearch
Horizontal Scale Topic/Partition RDD/Partition Index/Shard (default 5 primary shards per index)
Resource/cluster manager Zookeeper YARN, Mesos, or standalone Master Node
Cluster node Broker Worker Node
Execution unit Partition Leader Executor Lucene Index
Core
RDD Inverted index
Applications Producer/Consumer Driver Search
Replication Follower (ISR)
Replica shard


Kafka:



Spark:


Elasticsearch:






Sunday, June 23, 2019

Microservices


12 factor app pattern

Four-tier application architecture

Two key attributes of microservice architecture:

1.     Loosely coupled:  means that you can update the services independently; updating one service doesn’t require changing any other services.
2.     bounded context: A microservice with correctly bounded context is self‑contained for the purposes of software development. You can understand and update the microservice’s code without knowing anything about the internals of its peers, because the microservices and its peers interact strictly through APIs and so don’t share data structures, database schemata, or other internal representations of objects.

Best practices for designing a microservices architecture:

1.     Create a separate data store for each microservice.
2.     Keep code at a similar level of maturity: if you need to add or rewrite some of the code in a deployed microservice that is working well, create a new microservice for the new or changed code.  Then deploy and test the new one.  Then merge/combine them.
3.     Do a separate build for each microservice.
4.     Deploy in containers.
5.     Treat servers as stateless.





API Gateway

·       Ngnix
·       Netflix Zuul 1
·       Spring Cloud Gateway
·       Linkerd

Their performance comparison:

https://engineering.opsgenie.com/comparing-api-gateway-performances-nginx-vs-zuul-vs-spring-cloud-gateway-vs-linkerd-b2cc59c65369


OpenAPI

OpenAPI Specification (formerly known as Swagger spec) is the industry standard for defining REST APIs.

Sample:

    swagger: "2.0"
    info:
      title:
"Airport Codes"
      description:
"Get the name of an airport from its three-letter IATA code."
      version:
"1.0.0"
   
# This field will be replaced by the deploy_api.sh script.
    host:
"YOUR-PROJECT-ID.appspot.com"
    schemes:
      -
"https"
    paths:
     
"/airportName":
       
get:
          description:
"Get the airport name for a given IATA code."
          operationId:
"airportName"
          parameters:
            -
              name: iataCode
             
in: query
              required:
true
              type:
string
          responses:
           
200:
              description:
"Success."
              schema:
                type:
string
           
400:
              description:
"The IATA code is invalid or missing."

API spec edit APICURIO:






hapi-swagger: