액터를 사용한 신뢰성있는 메시지 전송 전략

Local전송 Vs Remote 전송

 Actor는 Local처리(전송)과 Remote 전송하는 사용방법이 거의 동일합니다.

...

• Remote의 경우 처리 대역폭에따라 네트워크 지연발생할수 있음
• Local에서는 제한없는 메시지 크기가 Remote에서는 크기제한 필요
• Remote의 경우 전송성능을 위해 TCP처리모듈, 바이너리데이터 해석을위해 Serialization 등의 모듈이 활용이되지만 신경쓸 필요없습니다.

Image Removed

• 선택이가능합니다.(옵션선택사항)
• Local에서의 전송장애는 실제 네트워크사용은 없기때문에 메모리(메모리풀)문제로 인해 발생할수 있으며, Remote의 경우는 네트워크 단절(Disconnect)에의해 발생할수 있습니다. 

Image Added

네트워크 전송에서는 Serialization/Deserializtion 과정을 통해 Object를 복원합니다. ( Object → Btyes → Object ) 

이기종간 개발 플래폼간에 메시지 클래스/구조체를 변환,복원하는 과정을 마샬링이라고도 합니다.

대표적 전송방법

...

• at-most-once delivery ( 한번에 한번씩 배달 )
• message ordering per sender - recceiver pair  ( 메시지 순서를 고려하여 작은단위 지속적 배달 )

...

AKKA자체로 만회할수 있는 것이 아닙니다. AKKA는 분산 컴퓨팅을 채택하고  메시지전달을통해

명시적으로 통신의 오류를 체크하기때문에 체크하며, 전달 보증을 위해 추상화된 거짓처리를 하지 않습니다. 

메시지 전달은 항상 보장되는게 아니며, AKKA는 메시지전달에 최소한 "보증"및 체크 수준을 제공해주고

사용자가 그것을 응용프로그램에 설계를 해야합니다.

이것은  Erlang에서 크게 성공을 거둔 기법입니다. ( http://erlang.org/faq/academic.html#idp32880720  - 10.9)

Higher-level abstractions

AKKA의 메시지는 강력하고 높은 수준의 추상화를 제공합니다.

메시지 전송순서(Message Ordering) 보장

•  원문 이해부터 하고 한글화할것... PSMON08 Sep 2017 

전송순서 체크 시나리오:Actor A1 sends messages 

CaseA

• Actor1이 Actor2에게 M1,

...

• M2,

...

• M3

...

• 메시지를 보낸다.

...

• Actor3가 Actor2에게 M4,M4,M5 메시지를 보낸다.

Image Added M5, M6 to A2.

This means that:

• If M1 is delivered it must be delivered before M2 and M3
• If M2 is delivered it must be delivered before M3
• If M4 is delivered it must be delivered before M5 and M6
• If M5 is delivered it must be delivered before M6
• A2 can see messages from A1 interleaved with messages from A3
• Since there is no guaranteed delivery, any of the messages may be dropped, i.e. not arrive at A2

Please note that this rule is not transitive:

Actor A sends message M1 to actor C

Actor A then sends message M2 to actor B

Actor B forwards message M2 to actor C

Actor C may receive M1 and M2 in any order

Causal transitive ordering would imply that M2 is never received before M1 at actor C (though any of them might be lost). This ordering can be violated due to different message delivery latencies when A, B and C reside on different network hosts, see more below.

Communication of failure

Please note, that the ordering guarantees discussed above only hold for user messages between actors. Failure of a child of an actor is communicated by special system messages that are not ordered relative to ordinary user messages. In particular:

Child actor C sends message M to its parent P

Child actor fails with failure F

Parent actor P might receive the two events either in order M, F or F, M

The reason for this is that internal system messages has their own mailboxes therefore the ordering of enqueue calls of a user and system message cannot guarantee the ordering of their dequeue times.

The Rules for In-App (Local) Message Sends

Be careful what you do with this section!

Relying on the stronger reliability in this section is not recommended since it will bind your application to local-only deployment: an application may have to be designed differently (as opposed to just employing some message exchange patterns local to some actors) in order to be fit for running on a cluster of machines. Our credo is "design once, deploy any way you wish", and to achieve this you should only rely on The General Rules.

Reliability of Local Message Sends

The Akka.NET test suite relies on not losing messages in the local context (and for non-error condition tests also for remote deployment), meaning that we actually do apply the best effort to keep our tests stable. A local Tell operation can however fail for the same reasons as a normal method call can on the CLR:

• StackOverflowException
• OutOfMemoryException
• other :SystemException

In addition, local sends can fail in Akka-specific ways:

• if the mailbox does not accept the message (e.g. full BoundedMailbox)
• if the receiving actor fails while processing the message or is already terminated

While the first is clearly a matter of configuration the second deserves some thought: the sender of a message does not get feedback if there was an exception while processing, that notification goes to the supervisor instead. This is in general not distinguishable from a lost message for an outside observer.

Ordering of Local Message Sends

Assuming strict FIFO mailboxes the abovementioned caveat of non-transitivity of the message ordering guarantee is eliminated under certain conditions. As you will note, these are quite subtle as it stands, and it is even possible that future performance optimizations will invalidate this whole paragraph. The possibly non-exhaustive list of counter-indications is:

• Before receiving the first reply from a top-level actor, there is a lock which protects an internal interim queue, and this lock is not fair; the implication is that enqueue requests from different senders which arrive during the actor's construction (figuratively, the details are more involved) may be reordered depending on low-level thread scheduling. Since completely fair locks do not exist on the CLR this is unfixable.

• The same mechanism is used during the construction of a Router, more precisely the routed ActorRef, hence the same problem exists for actors deployed with Routers.

• As mentioned above, the problem occurs anywhere a lock is involved during enqueueing, which may also apply to custom mailboxes.

This list has been compiled carefully, but other problematic scenarios may have escaped our analysis.

How does Local Ordering relate to Network Ordering

The rule that for a given pair of actors, messages sent directly from the first to the second will not be received out-of-order holds for messages sent over the network with the TCP based Akka.NET remote transport protocol.

As explained in the previous section local message sends obey transitive causal ordering under certain conditions. This ordering can be violated due to different message delivery latencies. For example:

Actor A on node-1 sends message M1 to actor C on node-3

Actor A on node-1 then sends message M2 to actor B on node-2

Actor B on node-2 forwards message M2 to actor C on node-3

Actor C may receive M1 and M2 in any order

It might take longer time for M1 to "travel" to node-3 than it takes for M2 to "travel" to node-3 via node-2.

Higher-level abstractions

Based on a small and consistent tool set in Akka's core, Akka.NET also provides powerful, higher-level abstractions on top it.

Messaging Patterns

As discussed above a straight-forward answer to the requirement of reliable delivery is an explicit ACK–RETRY protocol. In its simplest form this requires

• a way to identify individual messages to correlate message with acknowledgement
• a retry mechanism which will resend messages if not acknowledged in time
• a way for the receiver to detect and discard duplicates

The third becomes necessary by virtue of the acknowledgements not being guaranteed to arrive either. An ACK-RETRY protocol with business-level acknowledgements is supported by At least once delivery of the Akka.NET Persistence module. Duplicates can be detected by tracking the identifiers of messages sent via At least once delivery. Another way of implementing the third part would be to make processing the messages idempotent on the level of the business logic.

Event Sourcing

Event sourcing (and sharding) is what makes large websites scale to billions of users, and the idea is quite simple: when a component (think actor) processes a command it will generate a list of events representing the effect of the command. These events are stored in addition to being applied to the component's state. The nice thing about this scheme is that events only ever are appended to the storage, nothing is ever mutated; this enables perfect replication and scaling of consumers of this event stream (i.e. other components may consume the event stream as a means to replicate the component's state on a different continent or to react to changes). If the component's state is lost—due to a machine failure or by being pushed out of a cache—it can easily be reconstructed by replaying the event stream (usually employing snapshots to speed up the process). Event-sourcing is supported by Akka.NET Persistence.

Mailbox with Explicit Acknowledgement

By implementing a custom mailbox type it is possible to retry message processing at the receiving actor’s end in order to handle temporary failures. This pattern is mostly useful in the local communication context where delivery guarantees are otherwise sufficient to fulfill the application’s requirements.

Dead Letters

Messages which cannot be delivered (and for which this can be ascertained) will be delivered to a synthetic actor called /deadLetters. This delivery happens on a best-effort basis; it may fail even within a single application in the local machine (e.g. during actor termination). Messages sent via unreliable network transports will be lost without turning up as dead letters.

What Should I Use Dead Letters For?

The main use of this facility is for debugging, especially if an actor send does not arrive consistently (where usually inspecting the dead letters will tell you that the sender or recipient was set wrong somewhere along the way). In order to be useful for this purpose it is good practice to avoid sending to DeadLetters where possible, i.e. run your application with a suitable dead letter logger (see more below) from time to time and clean up the log output. This exercise—like all else—requires judicious application of common sense: it may well be that avoiding to send to a terminated actor complicates the sender's code more than is gained in debug output clarity.

The dead letter service follows the same rules with respect to delivery guarantees as all other message sends, hence it cannot be used to implement guaranteed delivery.

How do I Receive Dead Letters?

An actor can subscribe to class Akka.Actor.DeadLetter on the event stream, see Event stream for how to do that. The subscribed actor will then receive all dead letters published in the (local) system from that point onwards. Dead letters are not propagated over the network, if you want to collect them in one place you will have to subscribe one actor per network node and forward them manually. Also consider that dead letters are generated at that node which can determine that a send operation is failed, which for a remote send can be the local system (if no network connection can be established) or the remote one (if the actor you are sending to does not exist at that point in time).

Dead Letters Which are (Usually) not Worrisome

• M1 메시지는 M2,M3 보다 먼저도착 되는게 보장된다.
• M5 메시지는 M6보다 먼저 도착 되는게 보장된다.
• 보낸 Actor가 틀린 메시지 M2,M5는 누가 먼저 도착할지 모른다.
• 어느 메시지이던, 전송 보장이 없기때문에 드롭이 될수가 있다.

Case B

Image Added

• ActorA가 M1메시지를 ActorC 에게보낸다
• ActorA가 ActorB에게 M2메시지를 보내고, ActorB는 다시 ActorC에게 M2메시지를 포워드해서 보낸다.
  
  

장애감지 메시지(Communication of failure)

Image Added

• Child Actor인 B가 부모액터A에게 메시지 M을 전송한다.
• Child Actor B에 어떠한 F(장애메시지)가 발생하여 ActorA에게 전송되었다

 이경우 M,F 또는 F,M 순서로 받을수 있으며 순서가 보장되지 않습니다. 부모액터는 자식의 장애를 감시하고

처리하는 메카니즘이 있어서 관련된 장애메시지를 받고 처리가능하지만

일반 메시지와 분리된 큐에서 작동을 하기때문에 이 둘의 메시지에대해서는 순서보장이 되지 않습니다.

Messaging Patterns

안정적 메시지 전달요구사항에 대응이 가능해야합니다.

• 개별 메시지를 식별하여 메시지와 응답을 연관시키는법
• 시간내에 확인되지 않으면 다시보내는 재시도 메카니즘
• 수신기가 중복을 감지하고 버릴수 있는방법

Dead Letters(전송실패 메시지)

전송시도후 어떠한 이유로 전송이실패가 될시, 전송실패의 이유를 최선의 노력으로 

DeadLetter라는 액터로 전달을 합니다. 신뢰할수없는 네트워크 전송은 데드레터로 표시되지않고

손실됩니다.

데드레터는 어떻게 활용이 되는가?

데드레터의 주요 용도는 디버깅을 위한 것입니다. 목적지를 잘못설정하거나, 방화벽으로 막힌 목적지에

보낸다거나할때 DeadLetter는 이러한 사실을 감지하게 됩니다. 여러가지 문제를 제거하여

디버그 출력 선명도를 높일수 있지만,  데드레터를 피하려고 종료된 액터로 메시지전송을 일부러 

막는것은 송신자의 코드가 복잡해질수 있음으로 신중해야합니다. 

일반적으로 액터종료가 의도되는 사항이라면, 종료처리를 통해 이사실을 알리고 전송을 중단 시켜야할것입니다.

데드레터를 원격지에서 수신받을수 있는가?

데드레터는 네트워크를 통해 전달되지 않음으로, 한위치에 수집하려면 네트워크 하나당 액터를 구독하고

포워드를 하여야합니다. 또한 수신받는 원격지또한 지점간 네트워크 장애가 발생할수 있음으로

해당 데드레터를 수신받지 못할수도 있습니다Every time an actor does not terminate by its own decision, there is a chance that some messages which it sends to itself are lost. There is one which happens quite easily in complex shutdown scenarios that is usually benign: seeing a Akka.Dispatch.Terminate message dropped means that two termination requests were given, but of course only one can succeed. In the same vein, you might see Akka.Actor.Terminated messages from children while stopping a hierarchy of actors turning up in dead letters if the parent is still watching the child when the parent terminates.