Ergo Framework - is an implementation of ideas, technologies, and design patterns from the Erlang world in the Go programming language. It is built on the actor model, network transparency, and a set of ready-to-use components for development. This makes it significantly easier to create complex and distributed solutions while maintaining a high level of reliability and performance.

Node

The management of starting and stopping processes (actors), as well as routing messages between them, is handled by the node. The node is also responsible for network communication. If a message is sent to a remote process, the node automatically establishes a network connection with the remote node where the process is running, thus ensuring network transparency.

Actors

Actors in Ergo Framework are lightweight processes that live on top of goroutines. Interaction between processes occurs through message passing. Each process has a mailbox with multiple queues to prioritize message handling. Actors can exchange asynchronous messages and also make synchronous requests to each other.

Networking

Each node has a built-in Service Discovery function. Typically, the first node launched on a host becomes the registrar for other nodes running on the same host. Upon registration, each node reports its name, the port number for incoming connections, and a set of additional parameters.

Network transparency in Ergo Framework is achieved through the ENP (Ergo Network Protocol) and EDF (Ergo Data Format). These are based on ideas from Erlang's network stack but are not compatible with it. Therefore, to communicate with Erlang nodes, an additional Erlang package must be used.

To maximize performance in network message exchange, Ergo Framework uses a pool of multiple TCP connections combined into a single logical network connection.

Performance

Ergo Framework demonstrates high performance in local message exchange due to the use of lock-free queues within the process mailbox and the efficient use of goroutines to handle the process itself. A goroutine is only activated when the process receives a message; otherwise, the process remains idle and does not consume CPU resources.

You can evaluate the performance yourself using the provided benchmarks https://github.com/ergo-services/benchmarks.

Requirements

In the development of Ergo Framework, we adhere to the concept of zero dependencies. All functionality is implemented using only the standard Go library. The only requirement for using Ergo Framework is the version of the Go language. Starting from version 3.0, Ergo Framework depends on Go 1.20 or higher.

To create fault-tolerant applications, Ergo Framework introduces a process structuring model. The core idea of this model is to divide processes into two types:

• worker

• supervisor

Worker processes perform computations, while supervisors are responsible solely for managing worker processes.

In Ergo Framework, worker processes are actors based on act.Actor. Supervisors, on the other hand, are actors based on act.Supervisor. The role of act.Supervisor is to start child processes and restart them according to the chosen restart strategy. Several restart strategies are available for this purpose. A child process can be not only an actor based on act.Actor but also a supervisor based on act.Supervisor. This allows you to form a hierarchical structure for managing processes. Thanks to this approach, your solution becomes more reliable and fault-tolerant.

A Node is the core of the service you create using Ergo Framework. This core includes:

• Process Management Subsystem: handles the starting/stopping of processes, and the registration of process names/aliases.

• Message Routing Subsystem: manages the routing of asynchronous messages and synchronous requests between processes.

• Pub/Sub Subsystem: powers the event, link, and monitor functionalities, enabling distributed event handling.

• Network Stack: provides service discovery and network transparency, facilitating seamless communication between nodes.

• Logging Subsystem

Starting a node

To start a node, the first step is to define its name. The name consists of two parts: <name>@<hostname>, where the hostname determines on which network interface the port for incoming connections will be opened.

The node's name must be unique on the host. This means that two nodes with the same name cannot be running on the same host.

Below is an example code for starting a node:

import (
    "ergo.services/ergo"
    "ergo.services/ergo/gen"
)

func main() {
    name := "example@localhost"

    // Start node. Returns gen.Node interface
    node, err := ergo.StartNode(name, opts)
    if err != nil {
        panic(err)
    }
    
    node.Wait()
}

If the gen.NodeOptions.Applications option includes applications when starting the node, they will be automatically loaded and started. However, if any application fails to start, the ergo.StartNode(...) function will return an error, and the node will shut down. Upon a successful start, this function returns the gen.Node interface.

You can also specify environment variables for the node using the Env option in gen.NodeOptions. All processes started on this node will inherit these variables. The gen.Node interface provides the ability to manage environment variables through methods like EnvList(), SetEnv(...), and Env(...). However, changes to environment variables will only affect newly started processes. Environment variable names are case-insensitive.

Additionally, the gen.Node interface provides the following methods:

• Starting/Stopping Processes: methods like Spawn(...), SpawnRegister(...) allow you to start processes, while Kill(...) and SendExit(...) are used to stop them.

• Retrieving Process Information: use the ProcessInfo(...) method to get information about a process, and the MetaInfo(...) method to retrieve information about meta-processes.

• Managing the Node's Network Stack: access the gen.Network interface through the Network() method to manage the node's network operations.

• Getting Node Uptime: the Uptime() method provides the node's uptime in seconds.

• General Node Information: use the Info() method to retrieve general information about the node.

• Sending Asynchronous Messages: the Send(...) method allows you to send asynchronous messages to a process.

The full list of available methods for the gen.Node interface can be found in the reference documentation.

Process management

The mechanism for starting and stopping processes is provided by the node's core. Each process is assigned a unique identifier, gen.PID, which facilitates message routing.

The node also allows you to register a name associated with a process. You can register such a name at the time of process startup by using the SpawnRegister method and specifying the desired name. Alternatively, you can use the RegisterName method from the gen.Node or gen.Process interfaces to assign a name to an already running process.

A process can only have one associated name. If you want to change a process's name, you must first unregister the existing name using the UnregisterName method from the gen.Node or gen.Process interfaces. Upon success, this method returns the gen.PID of the process previously associated with that name.

A process can be terminated by the node by sending it an exit signal. This is done via the SendExit method in the gen.Node interface. If necessary, the node can also forcefully stop a process using the Kill method provided by the gen.Node interface.

Message routing

One of the key responsibilities of a node is message routing between processes. This routing is transparent to both the sender and the receiver processes. If the recipient process is located on a remote node, the node will automatically attempt to establish a network connection to the remote node and deliver the message to the recipient. This is the network transparency provided by the Ergo Framework—you don't need to worry about how to send the message or how to encode it, as the node handles all of this automatically and transparently for you.

Message routing is not limited to the gen.PID process identifier. A message can also be addressed using:

• Local Process Name (gen.Atom): messages can be sent to a local process by referencing its registered name.

• gen.Alias: this is a unique identifier that can be created by the process itself. It is often used for meta-processes or as a process temporary identifier. You can read more about this feature in the Process section.

• gen.ProcessID: This is used for sending messages by the recipient process's name. The structure contains two fields—Name and Node. It is a convenient way to send a message to a process on a remote node when you do not know the gen.PID or gen.Alias of the remote process.

Through these flexible options, the Ergo Framework provides a robust and seamless message routing system, simplifying communication between processes whether they are local or remote.

Pub/Sub subsystem

The core of the node implements a publisher/subscriber mechanism, which serves as the foundation for process linking, monitoring, and connections with other nodes.

• Monitoring Functionality: this allows any process to monitor other processes or nodes. In the event that a monitored process stops or the connection to a node is lost, the process that created the monitor will receive a gen.MessageDownPID or gen.MessageDownNode message, respectively.

• Linking Functionality: similar to monitoring, linking differs in that when a linked process terminates or the connection to a node is lost, the process that created the link will receive an exit signal, causing it to stop as well.

• Event System: the publisher/subscriber mechanism is also used in the events functionality. It allows any process to register its own event type and act as a producer of those events, while other processes can subscribe to those events.

Thanks to network transparency, the pub/sub subsystem works not only for local processes but also for remote ones.

You can read more about these features in the sections on Links and Monitors and Events.

Network stack

The process of sending a message to a remote node involves several steps:

1. Connection Establishment: if a connection to the remote node has not yet been created, the node queries the registrar on the host where the target node (owner of the recipient gen.PID, gen.ProcessID, or gen.Alias) is running. The registrar returns the port number on which the target node accepts incoming connections. The connection to this node is then established. This connection remains active until one of the nodes explicitly closes it by calling Disconnect from the gen.RemoteNode interface.

2. Message Encoding: the message is encoded into the binary EDF format.

3. Data Compression: if compression was enabled for the sender process, the binary data is compressed.

4. Message Transmission: the message is sent over the network using the ENP protocol.

5. Message Decoding and Delivery: remote node automatically decodes the received message and ensures its delivery to the recipient process's mailbox.

For more detailed information on network interactions, refer to the Network Stack section.

Node shutdown

You can stop a node using the Stop() or StopForce() methods from the gen.Node interface.

• Stop: all processes will be sent an exit signal with the reason gen.TerminateReasonShutdown from the parent process, and the node will wait for all processes to terminate. Once all processes have stopped, the node's network stack and the node itself will be shut down.

• StopForce: in the case of a forced shutdown, all processes will be terminated using the Kill method from the gen.Node interface, without waiting for their graceful shutdown.

Data types

gen.Atom

This type is an alias for the string type. It was introduced as a specialized string that allows differentiation between regular strings and those used for node names or process names. In the network stack, this type is also handled separately and is actively used in the Atom-cache and Atom-mapping mechanisms. When printed, values of this type are enclosed in single quotes.

fmt.Printf("%s", gen.Atom("hello"))
...
'hello'

gen.PID

This type is used as a process identifier. It is a structure containing several fields: the node name Node, a unique sequential number within the node ID, and a Creation field.

When printed as a string, this type is transformed into the following representation:

pid := gen.PID{Node:"t1node@localhost", ID:1001, Creation:1685523227}
fmt.Printf("%s", pid)
...
<90A29F11.0.1001>

The node name is encoded into a hash using the CRC32 algorithm.

gen.Ref

The node can generate unique identifiers. For this purpose, the gen.Node interface provides the MakeRef method. It returns a guaranteed unique value of type gen.Ref. These values are used as unique request identifiers in synchronous calls and as unique tokens when registering a gen.Event by a producer process.

When printed as a string, the value is represented as:

ref := gen.Ref{
    Node:"t1node@localhost", 
    Creation:1685524098, 
    ID:[3]uint32{0x1f4c2, 0x5d90, 0x0},
}
fmt.Printf("%s", ref)
...
Ref#<90A29F11.128194.23952.0>

gen.ProcessID

This type is used as a process identifier with an associated name. It is a structure containing two fields: the process name Name and the node name Node.

When printed as a string, the node name is transformed into a hash using the CRC32 algorithm, resulting in the following representation:

process := gen.ProcessID{Name:"example", Node:"t1node@localhost"}
fmt.Printf("%s", process)
...
<90A29F11.'example'>

gen.Alias

This type is an alias for the gen.Ref type. Values of type gen.Alias are used as temporary process identifiers, created using the CreateAlias method in the gen.Process interface, as well as identifiers for meta-processes.

When printed as a string, the representation is similar to gen.Ref, but with a different prefix:

alias := gen.Alias{
    Node:"t1node@localhost", 
    Creation:1685524098, 
    ID:[3]uint32{0x1f4c2, 0x5d90, 0x0},
}
fmt.Printf("%s", ref)
...
Alias#<90A29F11.128194.23952.0>

gen.Event

Values of this type are used when subscribing to events through the MonitorEvent and LinkEvent methods. This type is a structure similar to gen.ProcessID, containing two fields: Name and Node.

When printed as a string, the representation is similar to gen.ProcessID, but with an added prefix:

event := gen.Event{Name:"event1", Node:"t1node@localhost"}
fmt.Printf("%s", event)
...
Event#<90A29F11:'event1'>

gen.Env

This type is an alias for the string type. It is used for the names of environment variables for nodes and processes. In Ergo Framework, environment variable names are case-insensitive.

When printed as a string, the value of this type is converted to uppercase:

env := gen.Env("name1")
fmt.Printf("%s", env)
...
NAME1

Interfaces

gen.Node

To start a node, use the function ergo.StartNode(...). If the node starts successfully, it returns the gen.Node interface. This interface provides a set of functions for interacting with the node. The full list of methods can be found in the reference documentation.

gen.Process

This type is an interface to a process object. In Ergo Framework, this interface is embedded in the actor act.Actor, so all methods of this interface become available to objects based on act.Actor (and its derivatives).

type myActor struct {
    act.Actor
}

func (a *myActor) Init(args ...any) error {
    // Sending a message to itself using methods PID and Send 
    // that belong to the embedded gen.Process interface
    a.Send(a.PID(), "hello")
}

The full list of available methods for the gen.Process interface can be found in the reference documentation.

gen.Network

You can access this interface using the Network method of the gen.Node interface. This interface provides a set of methods for managing the node's network stack. The full list of available methods for this interface can be found in the reference documentation.

gen.RemoteNode

This interface allows you to retrieve information about a remote node with which a connection has been established, as well as to spawn processes (using the Spawn and SpawnRegister methods) and start applications on it (using the ApplicationStart* methods). You can access this interface through the GetNode and Node methods of the gen.Networkinterface.

Meta-processes are designed to integrate synchronous objects into the asynchronous model of Ergo Framework. Although they share some similarities with regular processes, they have a different nature and operational characteristics:

• Meta-Process Identifier: meta-process has a process identifier (of type gen.Alias) and is associated with its parent process. This allows message routing and synchronous requests to be directed to the meta-process.

• Sending Messages: meta-process can send asynchronous messages to other processes or meta-processes (including remote ones) using the Send method in the gen.MetaProcess interface. However, the sender will always appear as the parent process's gen.PID.

• Handling Messages: meta-processes can handle asynchronous messages from other processes or meta-processes via the HandleMessage callback method of the gen.MetaBehavior interface.

• Handling Synchronous Requests: can handle synchronous requests from other processes (including remote ones) using the HandleCall callback method of the gen.MetaBehavior interface.

• Spawning Other Meta-Processes: can spawn other meta-processes using the Spawn method in the gen.MetaProcess interface.

• Linking and Monitoring: other processes (including remote ones) can create a link or monitor with the meta-process using the LinkAlias and MonitorAlias methods of the gen.Process interface, referencing the meta-process’s gen.Alias.

Despite these similarities with regular processes, meta-processes have distinct characteristics:

• Concurrency: meta-process operates with two goroutines. The main goroutine starts when the meta-process is created and handles operations on the synchronous object. The termination of this goroutine leads to the termination of the meta-process. The auxiliary goroutine is launched to handle incoming messages from other processes or meta-processes, and it shuts down when there are no more messages in the meta-process’s mailbox. Therefore, access to the meta-process object’s data is concurrent, as it can be accessed by both goroutines simultaneously. This concurrency must be managed carefully when implementing your own meta-process.

• Parent Process Dependency: a meta-process can only be started by a process (or another meta-process) using the SpawnMeta method in the gen.Process interface (or the Spawn method in the gen.MetaProcessinterface).

• No Own Environment Variables: meta-processes do not have their own environment variables. The Env and EnvList methods of the gen.MetaProcess interface return the environment variables of the parent process.

• Limitations: meta-processes cannot make synchronous requests, create links, or establish monitors themselves.

• Termination with Parent: when the parent process terminates, the meta-process is also automatically terminated.

Meta-processes offer a way to integrate synchronous objects into the asynchronous system while maintaining compatibility with the process communication model. However, due to their concurrent nature and relationship with the parent process, they require careful handling in certain scenarios.

Ready-to-Use Meta-Processes

Ergo Framework provides several ready-to-use implementations of meta-processes:

• meta.TCP: allows you to launch a TCP server or create a TCP connection.

• meta.UDP: for launching a UDP server.

• meta.Web: for launching an HTTP server and creating an http.Handler based on a meta-process.

• WebSocket Meta-Process: for working with WebSockets, a WebSocket meta-process is available. Since its implementation has a dependency on github.com/gorilla/websocket, it is provided as a separate package: ergo.services/meta/websocket.

These meta-processes offer pre-built solutions for integrating common networking and web functionality into your application.

Meta-process starting

To start a meta-process, the gen.Process interface provides the SpawnMeta method:

If a meta-process needs to spawn other meta-processes, the gen.MetaProcess interface implements the Spawn method:

In the gen.MetaOptions options, you can configure:

• MailboxSize: size of the meta-process’s mailbox for incoming messages.

• SendPriority: priority for messages sent by the meta-process.

• LogLevel: the logging level for the meta-process.

Upon successful startup, these methods return the meta-process identifier, gen.Alias. This identifier can be used to:

• Send asynchronous messages via the SendAlias method in the gen.Process interface.

• Make synchronous requests using the CallAlias method in the gen.Process interface.

• Create links or monitors with the meta-process using the LinkAlias or MonitorAlias methods in the gen.Processinterface.

Meta-process termination

To stop a meta-process, you can send it an exit signal using the SendExitMeta method of the gen.Process interface. When the meta-process receives this signal, it triggers the callback method Terminate from the gen.MetaBehaviorinterface.

Additionally, a meta-process will be terminated in the following cases:

• The parent process has terminated.

• The main goroutine of the meta-process has finished (i.e., the Start method of the gen.MetaBehavior interface has completed its work).

• The HandleMessage (or HandleCall) callback method returned an error while processing an asynchronous message (or a synchronous request).

• A panic occurred in any method of the gen.MetaBehavior interface.

Implementing Your Own Meta-Process

To create a custom meta-process in Ergo Framework, you need to implement the gen.MetaBehavior interface. This interface defines the behavior of the meta-process and includes the following key methods:

• Start: this method is called when the meta-process starts. It is responsible for initializing the meta-process and running the main logic. The Start method runs in the main goroutine of the meta-process, and when it finishes, the meta-process is terminated.

• HandleMessage: this callback is invoked to handle incoming asynchronous messages from other processes or meta-processes. It runs in the auxiliary goroutine that processes the meta-process’s mailbox.

• HandleCall: this method processes synchronous requests (calls) from other processes or meta-processes. It is also part of the auxiliary goroutine that processes incoming messages.

• Terminate: called when the meta-process receives an exit signal or when its parent process terminates. It is responsible for cleanup and finalizing the meta-process before it is removed.

Here is an example of implementing a custom meta-process in Ergo Framework - demonstrates how to create a meta-process that reads data from a network connection and sends it to its parent process:

In this example, we embed the gen.MetaProcess interface within the myMeta struct. This allows us to use the methods of gen.MetaProcess (like Send, Parent, and Log) inside the callback methods defined by gen.MetaBehavior.

The HandleMessage method processes asynchronous messages sent to the meta-process. It checks the type of the message and takes appropriate actions, such as writing data to the connection. If the method returns an error, it causes the termination of the meta-process, triggering the Terminate method to handle the cleanup.

The Terminate method ensures that resources like the network connection are properly released when the meta-process ends. If the meta-process encounters an error in HandleMessage or HandleCall, or if any other termination condition occurs, the Terminate method will be called to finalize the shutdown process.

List of available methods in the gen.MetaProcess Interface:

In Ergo Framework, a flexible logging system is implemented that allows multiple loggers to exist within a node. These loggers can be configured to receive specific logging levels.

For logging purposes, the gen.Node and gen.Process interfaces provide the Log method. This method returns the gen.Log interface, which is used for logging messages within the framework.

type Log interface {
	Level() LogLevel
	SetLevel(level LogLevel) error

	Logger() string
	SetLogger(name string)

	Trace(format string, args ...any)
	Debug(format string, args ...any)
	Info(format string, args ...any)
	Warning(format string, args ...any)
	Error(format string, args ...any)
	Panic(format string, args ...any)
}

Using the gen.Log interface, you can manage the logging level not only for the node but also for each individual process (including meta-processes). There are six standard logging levels that can be set using the SetLevel method:

• gen.LogLevelDebug

• gen.LogLevelInfo

• gen.LogLevelWarning

• gen.LogLevelError

• gen.LogLevelPanic

• gen.LogLevelDisabled

Additionally, there are specialized logging levels:

• gen.LogLevelDefault: This is the default logging level. When a node starts with this level, it defaults to gen.LogLevelInfo. For applications or processes, the logging level inherits from the node. Meta-processes inherit the logging level of their parent process.

• gen.LogLevelTrace: This level is used exclusively for deep debugging. To avoid accidental activation, this level cannot be set through the SetLevel method of the gen.Log interface. Instead, it can only be enabled during the startup of a node or process through gen.NodeOptions.Log.Level or gen.ProcessOptions.LogLevel.

By default, the logging level for the node is set to gen.LogLevelInfo. Processes inherit the node's logging level at startup unless explicitly set in gen.ProcessOptions.

The SetLogger method allows you to restrict logging to a specified logger. If no logger is specified, the log message will be delivered to all registered loggers. You can retrieve the list of registered loggers using the Loggers method of the gen.Node interface.

The gen.LogLevelDisabled level can be used to temporarily disable logging, either for the entire node or a specific process.

Furthermore, the gen.Node interface provides two methods for process-specific logging control:

• ProcessLogLevel(pid gen.PID): Retrieves the current logging level of a specific process.

• SetProcessLogLevel(pid gen.PID, level gen.LogLevel): Sets the desired logging level for a specific process.

Logger

By default, a node in Ergo Framework uses a standard logger, and its parameters can be configured during node startup via gen.NodeOptions.Log.DefaultLogger. You have the option to disable the default logger and use a process-based logger or integrate one (or more) loggers from an additional logging library.

If you wish to create your own logger, you simply need to implement the gen.LoggerBehavior interface. This interface defines the behavior of a logger, allowing you to customize the logging system according to your application's requirements. By implementing this interface, you can control how log messages are processed, formatted, and routed, providing flexibility beyond the default logging system.

type LoggerBehavior interface {
	Log(message MessageLog)
	Terminate()
}

To add a new logger, the gen.Node interface provides two methods:

• LoggerAdd(name string, logger gen.LoggerBehavior, filter ...gen.LogLevel): This method allows you to add a logger (an object implementing the gen.LoggerBehavior interface) to the node's logging system.

• LoggerAddPID(pid gen.PID, name string, filter ...gen.LogLevel): This method adds a process as a logger. In this case, the node sends log messages to the specified process. The act.Actor class provides a callback method HandleLog for handling such log messages.

Both methods accept a filter argument, which allows you to specify a set of log levels that the logger should handle. For example, if you want your logger to only receive messages at the gen.LogLevelPanic and gen.LogLevelInfo levels, you can list them in the filter:

node.LoggerAdd("my logger", myLogger, gen.LogLevelPanic, gen.LogLevelInfo)

If the filter argument is not specified, the default set of log levels, gen.DefaultLogFilter, will be used:

[]gen.LogLevel{
    LogLevelTrace,
    LogLevelDebug,
    LogLevelInfo,
    LogLevelWarning,
    LogLevelError,
    LogLevelPanic,
}

The logging methods of the gen.LoggerBehavior interface will only be invoked for messages that match the specified logging levels.

Reusing a logger name is not allowed. If a logger with the same name already exists, the LoggerAdd or LoggerAddPIDfunction will return the error gen.ErrTaken. To resolve this, you must first remove the existing logger from the system using the appropriate method: LoggerDelete or LoggerDeletePID from the gen.Node interface.

When a logger is removed from the system, the Terminate callback method of the gen.LoggerBehavior interface is called.

Default logger

When starting a node, you can specify logging parameters using the gen.NodeOptions.Log.DefaultLogger option. These parameters will be applied to the default logger, which outputs all log messages in text format to a specified output (by default, os.Stdout). With these options, you can configure:

• Output: The interface used for log message output. To direct log messages to both standard output and a file simultaneously, you can use the io.MultiWriter function from Golang's standard library:

  Copy

  var options gen.NodeOptions
  logFile, _ := os.Create("out.log")
  options.Log.Logger.Output = io.MultiWriter(os.Stdout, logFile)

• TimeFormat: Defines the format for the timestamp of log messages. You can use any existing format, such as time.DateTime (see time package constants), or define your own custom format. By default, the timestamp is output in nanoseconds.

• IncludeBehavior: Adds the behavior of the process or meta-process to the log message.

• IncludeName: Adds the registered name of the process to the log message, if it exists.

• Filter: Specifies the logging levels that the default logger will handle. Leave this field empty to use the default set of levels (gen.DefaultLogFilter).

• Disable: Disables the default logger.

Here is an example of how the standard logger's output might look with these options applied:

TimeFormat: time.DateTime
IncludeBehavior: true
IncludeName: true

func (a *MyActor) Init(args ...any) error {
    ...
    a.Log().Info("starting my actor")
    return nil
}
...
pid, err := node.SpawnRegister("example", factoryMyActor, gen.ProcessOptions{})
node.Log().Info("started MyActor with PID %s", pid)
...
2024-07-31 07:53:57 [info] <6EE4478D.0.1017> 'example' main.MyActor: starting my actor
2024-07-31 07:53:57 [info] 6EE4478D: started MyActor with PID <6EE4478D.0.1017>

The first field is the timestamp of the log message, the second field is the log level, and the third field is the source of the message. The sources can come from various interfaces:

• gen.Node.Log(): Uses the node's name in the form of a CRC32 hash. Example: 9F35C982

• gen.Process.Log(): The string representation of the process identifier gen.PID. Example: <9F35C982.0.1013>

• gen.MetaProcess.Log(): The string representation of the meta-process identifier gen.Alias. Example: Alias#<9F35C982.123663.24065.0>

• gen.Connection.Log(): String representations of the local and remote nodes. Example: 9F35C982-90A29F11

Process-logger

If you want to manage log message streams using the familiar actor model, any process in a running node can serve as a logger. In the act.Actor implementation, received log messages are passed as the message argument to the HandleLog callback method.

To add your actor to the node as a logger, use the LoggerAddPID method from the gen.Node interface:

func factoryMyLogger() gen.ProcessBehavior {
    return &MyLogger{}
}

type MyLogger struct {
    act.Actor
}

func (ml *MyLogger) HandleLog(message gen.MessageLog) error {
    switch m := message.Source.(type) {
    case gen.MessageLogNode:
        // handle message 
    case gen.MessageLogProcess:
        // handle message
    case gen.MessageLogMeta:
        // handle message
    case gen.MessageLogNetwork:
        // handle message
    }
    ...
    return nil
}
...
pid, err := node.Spawn(factoryMyLogger, gen.ProcessOptions{})
node.LoggerAddPID(pid)

Note that the LoggerAddPID method of the gen.Node interface is not available to a process during its initialization state, as the node has not yet recognized the process (see the Process section). Therefore, an actor cannot add itself as a logger within the Init callback method of the act.ActorBehavior interface.

When a logger process terminates, it will be automatically removed from the node's logging system.

Implementations of gen.Logger

To extend the capabilities of the Ergo Framework, two additional loggers have been implemented:

• colored: Enables colored highlighting for log messages in the console.

• rotate: Allows logging to a file with support for log file rotation at specified intervals.

You can disable the default logger and add both of these loggers, enabling colored output in the console while also saving logs to a file with rotation. You can find an example of using them in the demo project here.

A supervisor in Ergo Framework is a specialized actor responsible for managing child processes. Its main tasks include starting child processes, monitoring their life cycles, and restarting them if necessary according to the chosen restart strategy.

act.Supervisor implements the low-level gen.ProcessBehavior interface. Similar to act.Actor, it has the embedded gen.Process interface. This means that an object based on act.Supervisor has access to all the methods provided by the gen.Process interface.

To create a supervisor actor, you simply embed act.Supervisor into your object. Additionally, you need to create a factory function for the supervisor. Here is an example:

type MySupervisor struct {
    act.Supervisor
}

func factoryMySupervisor() gen.ProcessBehavior {
    return &MySupervisor{}
}

To work with your object, act.Supervisor uses the act.SupervisorBehavior interface. This interface defines a set of callback methods for initialization, handling asynchronous messages, and handling synchronous calls:

type SupervisorBehavior interface {
	gen.ProcessBehavior

	// Init invoked on a spawn Supervisor process. 
	// This is a mandatory callback for the implementation
	Init(args ...any) (SupervisorSpec, error)

	// HandleChildStart invoked on a successful child process starting if option EnableHandleChild
	// was enabled in gen.SupervisorSpec
	HandleChildStart(name gen.Atom, pid gen.PID) error

	// HandleChildTerminate invoked on a child process termination 
	// if option EnableHandleChild was enabled in gen.SupervisorSpec
	HandleChildTerminate(name gen.Atom, pid gen.PID, reason error) error

	// HandleMessage invoked if Supervisor received a message sent with gen.Process.Send(...).
	// Non-nil value of the returning error will cause termination of this process.
	// To stop this process normally, return gen.TerminateReasonNormal or
	// gen.TerminateReasonShutdown. Any other - for abnormal termination.
	HandleMessage(from gen.PID, message any) error

	// HandleCall invoked if Supervisor got a synchronous request made with gen.Process.Call(...).
	// Return nil as a result to handle this request asynchronously and
	// to provide the result later using the gen.Process.SendResponse(...) method.
	HandleCall(from gen.PID, ref gen.Ref, request any) (any, error)

	// Terminate invoked on a termination supervisor process
	Terminate(reason error)

	// HandleInspect invoked on the request made with gen.Process.Inspect(...)
	HandleInspect(from gen.PID) map[string]string
}

The Init method is mandatory for implementation, while the other methods are optional. This method returns the supervisor specification act.SupervisorSpec, which includes the following options:

• Type: The type of supervisor (act.SupervisorType).

• Children: The list of child process specifications ([]act.SupervisorChildSpec). This parameter also determines the order of child process startups.

• Restart: Defines the restart strategy for child processes.

• EnableHandleChild: Enables the invocation of the callback methods HandleChildStart and HandleChildStop during the startup and shutdown of child processes.

• DisableAutoShutdown: Prevents the supervisor from stopping if all child processes terminate non-faultily.

Here is an example implementation of the act.SupervisorBehavior interface for the supervisor MySupervisor with one child process MyActor:

//
// Child process implementation
//

func factoryMyActor() gen.ProcessBehavior {
    return &MyActor{}
}

type MyActor struct {
    act.Actor
}

func (a *MyActor) Init(args ...any) error {
    a.Log().Info("starting child process MyActor")
    return nil
}

//
// gen.SupervisorBehavior implementation
//

func (s *MySupervisor) Init(args ...any) (act.SupervisorSpec, error) {
    s.Log().Info("initialize supervisor...")
    spec := act.SupervisorSpec{
        Children: []act.SupervisorChildSpec{
            act.SupervisorChildSpec{
                Name: "myChild",
                Factory: factoryMyActor,
            },
        }
    }
    return spec, nil
}
...
pid, err := node.Spawn(factoryMySupervisor, gen.ProcessOptions{})
node.Log().Info("MySupervisor is started succesfully with pid %s", pid)
...

Supervisor Types

The type of supervisor in the SupervisorSpec defines its behavior. In Ergo Framework, several supervisor types are implemented:

act.SupervisorTypeOneForOne

This is the default supervisor type. If the type is not specified in the SupervisorSpec, this one will be used. Upon startup, the supervisor sequentially starts child processes as specified in act.SupervisorSpec.Children. If an error occurs while starting the child processes, the supervisor process will terminate.

Each child process is launched with the associated name defined in the Name option of the child process specification act.SupervisorChildSpec. This means that only one process can be started for each child process specification. If this field is empty, the supervisor will use the Spawn method from the gen.Process interface. Otherwise, the SpawnRegister method will be used.

Stopping a child process triggers the restart strategy for that process. Depending on the restart strategy defined by act.SupervisorSpec.Restart.Strategy, the process will either be restarted or stopped. The restart strategy does not affect other processes.

The child process is restarted with the same set of arguments that were used in its previous startup.

The KeepOrder option in act.SupervisorSpec.Restart is ignored for this supervisor type. When a new child process specification is added via the AddChild method (provided by act.Supervisor), the supervisor automatically starts the new child process. When a child specification is disabled (using method) DisableChild, the supervisor stops the corresponding child process with the reason gen.TerminateReasonShutdown. The supervisor will automatically terminate if no child processes remain (with the reasons gen.TerminateReasonNormal or gen.TerminateReasonShutdown). You can enable the DisableAutoShutdown option in act.SupervisorSpec to prevent this.

act.SupervisorTypeOneForAll

When starting, this supervisor sequentially launches child processes according to the list of child process specifications. If an error occurs during the startup of child processes, the supervisor process terminates.

A child process is started with the associated name defined in the Name option of act.SupervisorChildSpec.

Stopping a child process in this supervisor triggers the restart strategy for all child processes according to act.SupervisorSpec.Restart.Strategy.

If the KeepOrder option is enabled, child processes are stopped in reverse order. Without KeepOrder, they are stopped simultaneously. After stopping, the supervisor restarts them in the specified order.

Each child process is restarted with the same arguments as in the previous run.

When a new child process specification is added (using AddChild), the supervisor automatically starts the child process. When disabling (DisableChild), the supervisor stops the corresponding child process with the reason gen.TerminateReasonShutdown.

The DisableAutoShutdown option allows the supervisor to continue running even if all child processes have stopped (terminated with gen.TerminateReasonNormal or gen.TerminateReasonShutdown).

act.SupervisorTypeRestForOne

When starting, this supervisor sequentially launches child processes according to the list of child process specifications. If an error occurs during the startup of child processes, the supervisor process terminates.

A child process is started with the associated name defined in the Name option of act.SupervisorChildSpec.

When a child process terminates, the restart strategy is activated. All child processes starting from the last one (according to the order in act.SupervisorSpec.Children) to the process that triggered the restart are stopped.

With the KeepOrder option enabled, child processes are stopped sequentially in reverse order. If the option is disabled, child processes stop simultaneously.

After all child processes are stopped, the supervisor restarts them according to the order specified in act.SupervisorSpec.Children.

Each child process is restarted with the same set of arguments used in the previous startup.

Each child process is launched with the associated name defined in the Name option of the child process specification act.SupervisorChildSpec.

act.SupervisorTypeSimpleOneForOne

This type of supervisor is a simplified version of act.SupervisorTypeOneForOne. It does not start child processes on startup. Instead, child processes are started using the StartChild method.

As in act.SupervisorTypeOneForOne, the termination of a child process triggers the restart strategy for that process. Depending on the strategy defined in act.SupervisorSpec.Restart.Strategy, the process will either be restarted or stopped. The restart strategy does not affect other processes.

Child processes in this supervisor are launched without an associated name, and multiple processes can be started from one child process specification.

The KeepOrder option in act.SupervisorSpec.Restart is ignored for this supervisor type.

The DisableAutoShutdown option is also ignored, and stopping all child processes does not result in the termination of the supervisor process.

Child Process Specification

This specification describes the parameters for starting a child process. This is done using act.SupervisorChildSpec:

• Name: The specification's name, used as the associated name for the process (except for act.SupervisorTypeSimpleOneForOne).

• Factory, Options, Args: Parameters needed to launch a new process.

• Significant: Determines the importance of the child process. The termination of a significant process may lead to the supervisor and all child processes stopping, depending on the restart strategy:

  • act.SupervisorStrategyTemporary: Termination of a significant child process leads to the termination of the supervisor and all children.

  • act.SupervisorStrategyTransient: Non-crash termination (e.g., gen.TerminateReasonNormal or gen.TerminateReasonShutdown) of a significant child process leads to the termination of the supervisor and all children, while a crash triggers the restart strategy.

  • act.SupervisorStrategyPermanent: The Significant field is ignored, and termination always triggers the restart strategy.

Additionally, the Significant field is ignored in supervisors of types act.SupervisorTypeOneForOne and act.SupervisorSimpleOneForOne.

Restart Strategy

The Restart option in act.SupervisorSpec is of type act.SupervisorRestart and defines parameters for the restart strategy:

• Strategy: Specifies the restart strategy type:

  • act.SupervisorStrategyTransient: A crash triggers a restart. If the process terminates normally (e.g., gen.TerminateReasonNormal or gen.TerminateReasonShutdown), the restart strategy is not activated. This is the default strategy.

  • act.SupervisorStrategyTemporary: No restart occurs, even in the event of a crash.

  • act.SupervisorStrategyPermanent: Any termination triggers a restart, ignoring DisableAutoShutdown.

• Intensity and Period: Define the maximum number of restart activations allowed within a given period (in seconds). If this limit is exceeded, the supervisor terminates all child processes and itself, with the reason act.ErrSupervisorRestartsExceeded.

• KeepOrder: Specifies the order in which child processes are stopped when the restart strategy is activated. If this option is enabled, processes will be stopped sequentially in reverse order. By default, this option is disabled, meaning that all child processes are stopped simultaneously. Processes are restarted only after all child processes have fully stopped (either all processes for act.SupervisorTypeAllForOne or part of the processes for act.SupervisorTypeRestForOne).

Methods of act.Supervisor

• StartChild(name gen.Atom, args...): Starts a child process by its specification name. If arguments (args) are provided, they are updated in the specification for future restarts.

• AddChild(child act.SupervisorChildSpec): Adds a child specification and automatically starts the process.

• EnableChild(name gen.Atom): Enables and starts a previously disabled child process.

• DisableChild(name gen.Atom): Disables the specification and stops the running process.

• Children(): Returns a list of []act.SupervisorChild with data on specifications and running processes, sorted according to the child process specification list.

During initialization, the supervisor defines the initial set of child processes, the supervisor type, and the restart strategy for the child processes. act.Supervisor provides several methods for managing child processes dynamically and obtaining information about running processes:

Ergo Framework introduces gen.CertManager to simplify TLS certificate management. It enables live certificate updates without restarting the application and can be used across all network components like node acceptors, web, or TCP servers.

Use the function gen.CreateCertManager(cert tls.Certificate) to create a certificate manager. The gen.CertManager interface provides the following methods:

• Update: Update the TLS certificate on the fly. All components using this CertManager will automatically receive the updated certificate.

• GetCertificateFunc: Returns a closure to retrieve the TLS certificate.

• GetCertificate: Returns the current TLS certificate.

Additionally, you can use the lib.GenerateSelfSignedCert function to generate a self-signed certificate.

import (
    ...
    "ergo.services/ergo"
    "ergo.services/ergo/gen"
    "ergo.services/ergo/lib"
    ...
)

var (
    Version = gen.Version{
        Name:    "DemoService",
	Release: "1.0",
    }
)

func main() {
    var options gen.NodeOptions
    ...
    cert, err := lib.GenerateSelfSignedCert(Version.String())
    if err != nil {
        panic(err)
    }
    options.CertManager = gen.CreateCertManager(cert)
    ...
    node, err := ergo.StartNode("node@localhost", options)
    if err != nil {
	panic(err)
    }

    node.Wait()
} 

The events mechanism in Ergo Framework is built on top of the pub/sub subsystem. It allows any process to become an event producer, while other processes can subscribe to these events. This enables flexible event-driven architectures, where processes can publish and consume events across the system.

Producer

To register an event (gen.Event), you use the RegisterEvent method available in the gen.Process or gen.Nodeinterface. When registering an event, you can configure the following parameters using the gen.EventOptions:

• Notify: This flag controls whether the producer should be notified about the presence or absence of subscribers for the event. If notifications are enabled, the process will receive a gen.MessageEventStart message when the first subscriber appears and a gen.MessageEventStop message when the last subscriber unsubscribes. This allows the producer to generate events only when there are active subscribers. If the event is registered using the RegisterEvent method of the gen.Node interface, this field is ignored.

• Buffer: This specifies how many of the most recent events should be stored in the event buffer. If this option is set to zero, buffering is disabled.

The RegisterEvent function returns a token of type gen.Ref upon success. This token is used to generate events. Only the process that owns the event, or a process that has been delegated the token by the event owner, can produce events for that registration.

To generate events, the gen.Process interface provides the SendEvent method. This method accepts the following arguments:

• name: The name of the registered event (gen.Atom).

• token: The key obtained during the event registration (gen.Ref).

• message: The event payload, which can be of any type.

To generate events, the gen.Node interface provides the SendEvent method. This method is similar to the SendEventmethod in the gen.Process interface but includes an additional parameter, gen.MessageOptions. This extra parameter allows for further customization of how the event message is sent, such as setting priority, compression, or other message-related options.

Consumer

To subscribe to a registered event, the gen.Process interface provides the following methods:

• LinkEvent: This method creates a link to a gen.Event. The process will receive an exit signal if the producer process of the event terminates or if the event is unregistered. To remove the link, use the UnlinkEvent method. When the link is created, the method returns a list of the most recent events from the producer's buffer.

• MonitorEvent: This method creates a monitor on a gen.Event. The process will receive a gen.MessageDownEventif the producer process terminates or if the event is unregistered. If the event is unregistered, the Reason field in gen.MessageDownEvent will contain gen.ErrUnregistered. To remove the monitor, use the DemonitorEventmethod.

Both methods accept an argument of type gen.Event. For an event registered on the local node, you only need to specify the Name field, and you can leave the Node field empty. This simplifies subscribing to local events while still providing flexibility for handling events from remote nodes.

Upon successfully creating a link or monitor, the function returns a list of events (gen.MessageEvent) provided by the producer process from its message buffer. Each event contains the following information: the event name (gen.Event), the timestamp of when the event was generated (obtained using time.Now().UnixNano()), and the actual event value sent by the producer process. If the list of events is empty, it means that either the producer process has not yet generated any events or the producer registered the event with a zero-sized buffer.

This meta-process allows you to launch a UDP server and handle incoming UDP packets as asynchronous messages of type meta.MessageUDP.

To create the meta-process, use the meta.CreateUDPServer function. This function takes meta.UDPServerOptions as an argument, which specifies the configuration options for the UDP server, such as the host, port, buffer size, and other relevant settings for managing UDP traffic:

• Host, Port: Set the port number and host name for the UDP server.

• Process: The name of the process that will handle meta.MessageUDP messages. If not specified, all messages will be sent to the parent process. You can also use an act.Pool process for distributing message handling.

• BufferSize: Sets the buffer size created for incoming UDP messages. For each message, a new buffer of this size is allocated. This field is ignored if BufferPool is provided.

• BufferPool: Defines a pool for message buffers. The Get method should return an allocated buffer of type []byte.

If the meta-process is successfully created, the UDP server will start, and the function meta.CreateUDPServer will return gen.MetaBehavior.

Next, the created meta-process must be started. It will handle incoming UDP packets and forward them to the process for handling. To start the meta-process, use the SpawnMeta method of the gen.Process interface.

If an error occurs when starting the meta-process, the UDP server created by the meta.CreateUDPServer function must be stopped using the Terminate method of the gen.MetaBehavior interface.

Stopping the meta-process results in the shutdown of the UDP server and the closing of the socket.

Sending a UDP-packet

To send a UDP packet, use the Send (or SendAlias) method from the gen.Process interface. The recipient should be the identifier gen.Alias of the UDP server meta-process. The message should be of type meta.MessageUDP. The meta.MessageUDP.ID field is ignored when sending (it is used only for incoming messages).

If a BufferPool was used in the UDP server options, the buffer in meta.MessageUDP.Data will be returned to the pool after the packet is sent.

Example:

To work with your object, act.WebWorker uses the act.WebWorkerBehavior interface. This interface defines a set of callback methods:

All methods in the act.WebWorkerBehavior interface are optional for implementation.

It is most efficient to use act.WebWorker in combination with act.Pool for load balancing when handling HTTP requests:

This implementation of the meta-process allows running external programs and communicating with them using stdin and stdout. Communication can be performed in both text and binary formats.

To create a meta.Port, you need to use the meta.CreatePort function. It takes meta.PortOptions as an argument, which allows specifying the following options:

• Cmd command to run

• Args arguments for the command

• Env environment variables for the program being executed

• EnableEnvMeta adds environment variables of the meta-process

• EnableEnvOS adds OS environment variables

• Tag can help distinguish between multiple meta-processes running with the same Cmd/Args

• Process The name of the process that will handle meta.MessagePort* messages. If not specified, all messages will be sent to the parent process. You can also use an act.Pool process for distributing message handling

• SplitFuncStdout to specify a split-function. (uses Scanner from the standard bufio package) for processing data from stdout in text mode. By default, data is split by lines

• SplitFuncStderr to specify a split-function for the data from stderr

Below is an example of creating and running the external program example-prog using a meta-process meta.Port:

Binary mode

The implementation of meta.Port supports binary mode when working with stdin/stdout. To enable it, use the meta.PortOptions.Binary.Enable option and specify the binary format parameters:

• ReadBufferSize sets the buffer size for reading. The default size is 8192

• ReadBufferPool defines a pool for message buffers. The Get method should return an allocated buffer of type []byte

• ReadChunk enables auto-chunking mode, which splits the incoming stream into chunks according to the specified parameters. For more details, see Auto chunking

• WriteBufferKeepAlive enables the software keep-alive mechanism – a specified value will be automatically sent at the interval defined in the option WriteBufferKeepAlivePeriod

When binary mode is enabled, the SplitFuncStdout option is ignored. However, stderr is processed in text mode, and you can use the SplitFuncStderr option.

Auto chunking

This feature allows automatically chunking the incoming stream in binary mode. The chunks can be of fixed length – to achieve this, you need to specify the chunk size using the FixedLength option. For data with dynamic length, you must specify header parameters that will define the chunk length.

To enable the auto-chunking feature in binary mode, use the meta.PortOptions.Binary.ReadChunk.Enable option, and specify the parameters for automatic data chunking using the following options:

• FixedLength allows you to set the length of chunks in the stdout. In this case, all Header* options will be ignored. Leave this field with a zero value to work with dynamically sized chunks

• HeaderSize specifies the size of the header for dynamically sized chunks

• HeaderLengthPosition specifies the position in the header where the length is stored

• HeaderLengthSize specifies the size of the length in bytes (1, 2 or 4)

• HeaderLengthIncludesHeader specifies whether the length value includes the header size

• MaxLength allows you to set the maximum chunk length. If this value is exceeded, the meta-process will terminate, and the launched program will also be automatically stopped. Leave this value as zero if no length limits are required

For instance: incoming stream consists of dynamically sized chunks with a 7-byte header, and the length is encoded as a 32-bit number at the 3rd byte position (counting from 0). The length value includes the header size: L = 7 + len(payload)

In this case, the auto-chunking options would be as follows

To handle synchronous HTTP requests with the actor model in Ergo Framework, they must be converted into asynchronous messages. For this purpose, two meta-processes have been created: meta.WebServer and meta.WebHandler. The first is used to start an HTTP server, and the second transforms synchronous HTTP requests into messages that are sent to a worker process. A worker process can be created using act.WebWorker, which processes the incoming messages asynchronously.

Creating and Spawning meta.WebHandler

You can create a meta.WebHandler using the meta.CreateWebHandler function. This function creates an object that implements the gen.MetaProcessBehavior and http.Handler interfaces.

It takes meta.WebHandlerOptions as an argument:

• Process: The name of the process to which transformed asynchronous meta.MessageWebRequest messages are sent. If not specified, these messages are sent to the parent meta-process.

• RequestTimeout: The time allowed for the process to handle the message, with a default of 5 seconds.

After successful creation, you need to start the meta-process using SpawnMeta from the gen.Process interface.

Once the meta-process is running, it can be used as an HTTP request handler

Creating and Spawning meta.WebServer

To create the meta.WebServer meta-process, use the meta.CreateWebServer function with the meta.WebServerOptions argument. These options allow you to set:

• Host: The interface on which the port will be opened for handling HTTP requests.

• Port: The port number.

• CertManager: Enables TLS encryption for the HTTP server. You can use the node's CertManager to activate the node's certificate by using the CertManager() method of the gen.Node interface.

• Handler: Specifies the HTTP request handler.

When the meta-process is created, the HTTP server starts. If the server fails to start, meta.CreateWebServer returns an error. After successful creation, start the meta-process using SpawnMeta(...) from the gen.Process interface.

Example:

For working with TCP connections, two types of meta-processes are implemented in Ergo Framework:

1. TCP Server Process: Responsible for creating a TCP server. It opens the socket and handles incoming connections.

2. TCP Connection Handler: Manages established TCP connections, whether they are incoming or client-initiated

Server

To create a TCP server meta-process, use the meta.CreateTCPServer function. It accepts meta.TCPServerOptions with the following parameters:

• Host, Port: Specify the interface and port for incoming connections.

• ProcessPool: Defines a list of worker processes to handle incoming connections. Each new connection is assigned a worker from the pool. If ProcessPool is not set, the parent process handles all connections. Using act.Pool in ProcessPool is not recommended to avoid packet mismanagement.

• CertManager: Enables TLS encryption.

• BufferSize/BufferPool: Manage incoming data buffers.

• KeepAlivePeriod/InsecureSkipVerify: Handle TCP keep-alive and TLS certificate verification.

After creating the meta-process, you should start it with SpawnMeta from the gen.Process interface. Example:

If starting the meta-process fails for any reason, you need to free up the port specified in meta.TCPServerOptions.Port by using the Terminate method of the gen.MetaBehavior interface.

Upon an incoming connection, the TCP server meta-process creates and starts a new meta-process to handle that connection.

Connection

To create a client TCP connection, use the meta.CreateTCPConnection function. It initiates a TCP connection and, if successful, creates a meta-process to manage it.

The function takes meta.TCPConnectionOption, similar to meta.TCPServerOptions, but instead of ProcessPool, it includes a Process field for specifying a process to handle TCP packets. If this field is not used, all data packets are sent to the parent process.

After a successful connection, start the meta-process with SpawnMeta. If the process fails, close the connection using the Terminate method of the gen.MetaBehavior interface..

Example:

TCP-connection handling

To handle a TCP connection, the meta-process communicates with either the parent or worker process based on the meta-process's launch options. Three types of messages are used:

• meta.MessageTCPConnect: Sent when the meta-process starts, containing the connection ID (ID), RemoteAddr, and LocalAddr information.

• meta.MessageTCPDisconnect: Sent when the TCP connection is disconnected, including cases where the meta-process is terminated.

• meta.MessageTCP: Used for both receiving and sending data.

If the meta-process cannot send a message to the worker process, the connection is terminated, and the meta-process stops working.

Sending TCP-packet

To send a message to a TCP connection, use the Send (or SendAlias) method from the gen.Process interface. The recipient should be the gen.Alias of the meta-process managing the connection. The message must be of type meta.MessageTCP. Note that the meta.MessageTCP.ID field is ignored during sending (it is only used for incoming messages).

An actor in Ergo Framework implements the low-level gen.ProcessBehavior interface. To launch a process based on act.Actor, you need to create an object with an embedded act.Actor and implement a factory function for it. For example:

act.Actor uses the act.ActorBehavior interface to interact with your object. This interface defines a set of callback methods:

All methods in the act.ActorBehavior interface are optional, allowing you to implement only the necessary callbacks in your object. Since act.Actor embeds the gen.Process interface, you can directly use its methods from within your actor object.

Example:

Process Initialization

Process initialization begins when act.Actor invokes the Init method of the act.ActorBehavior interface, passing the args provided during Spawn (or SpawnRegister). If Init completes successfully, the node registers the process, allowing it to receive messages or synchronous requests. If Init fails, Spawn (or SpawnRegister) returns the error. It's important to note that during initialization, the process is not yet registered with the node, limiting access to certain methods of the embedded gen.Process interface until the process is fully initialized.

Handling Asynchronous Messages and Synchronous Requests

The process mailbox contains several queues: Main, System, Urgent, and a special Log queue. In act.Actor, messages are processed in the following order: Urgent, System, Main, and then Log.

Asynchronous messages are sent using the Send method of the gen.Node or gen.Process interface. Upon receiving such a message, act.Actor calls the HandleMessage method of the gen.ActorBehavior interface.

For synchronous requests, the HandleCall callback method is invoked in an actor based on act.Actor. The result returned by this method is sent as the response to the request.

A process can send asynchronous messages to itself, but attempting to make a synchronous request to itself will return the error gen.ErrTimeout..

act.Actor also allows handling synchronous requests asynchronously. This feature lets you delegate the processing of synchronous requests to other processes or delay the response. To do this, the HandleCall method should return nil, and you should use the SendResponse method of the gen.Process interface to send the response. You must provide the gen.PID of the requesting process and the gen.Ref reference of the request.

If the actor process is registered as a logger, the HandleLog method is called when log messages (gen.MessageLog) are received, generated using the gen.Log interface. If the actor process has subscribed to events using the LinkEvent or MonitorEvent methods of gen.Process, the HandleEvent method is called when an event message is received.

Process Termination

To stop a process, simply return a non-nil error value from the message-handling callback. After termination, the Terminate callback method will be called, with the reason argument being the returned error value.

Additionally, you can terminate a process using predefined constants like:

• gen.TerminateReasonNormal for a normal (non-failure) termination.

• gen.TerminateReasonKill when the process is forcibly stopped using the Kill method from the gen.Node interface.

In Ergo Framework, two additional termination reasons are defined:

• gen.TerminateReasonPanic: Occurs when a panic happens during message processing.

• gen.TerminateReasonShutdown: This termination reason can be sent by the parent process or the node when the node is stopped using the StopForce method of the gen.Node interface. It is not considered a failure.

At the moment the Terminate callback is invoked, the process has already been removed from the node. Therefore, most methods of the gen.Process interface will return the gen.ErrNotAllowed error.

SplitHandle

The SplitHandle option allows an actor to call the callback methods of the act.ActorBehavior interface based on the process identifier used to send a message or make a synchronous call to your process. You can enable this option using the SetSplitHandle(split bool) method defined in act.Actor. To check the current value of this option, use the SplitHandle method. Here is an example usage:

If your process has a registered associated name and receives a message using that name, act.Actor will call the HandleMessageName callback. The name argument will be the value used to send the message. For synchronous requests, the HandleCallName callback will be invoked.

If a process alias (gen.Alias) was used, the callbacks HandleMessageAlias and HandleCallAlias will be triggered for messages and synchronous requests, respectively. This enables customized handling based on the way the message or request was sent.

TrapExit - handling exit-signal

In Ergo Framework, an exit signal can be sent to your process by another process or node using the SendExit method from the gen.Process or gen.Node interfaces. It can also be generated by the node if your process had a link created via the Link* methods in the gen.Process interface. By default, when an actor receives an exit signal, it terminates and calls the Terminate callback with the reason specified in the exit signal.

To prevent your actor from terminating upon receiving an exit signal, you can enable the option to intercept such signals. This can be done using the SetTrapExit(trap bool) method in act.Actor. The default value is false, but you can enable it at any time, for example, during initialization:

When the TrapExit option is enabled, act.Actor converts exit signals into standard gen.MessageExit* messages:

• gen.MessageExitPID: The source of the exit signal was a process.

• gen.MessageExitProcessID: The source was gen.ProcessID (a link created using LinkProcessID).

• gen.MessageExitAlias: The source was gen.Alias (link created using LinkAlias).

• gen.MessageExitEvent: The source was gen.Event (link created using LinkEvent).

• gen.MessageExitNode: The source was a network connection with a node (link created using LinkNode).

The act.Pool actor implements the low-level gen.ProcessBehavior interface and provides the "Pool of Workers" design pattern functionality. All asynchronous messages and synchronous requests sent to the pool process are redirected to worker processes in the pool.

To launch a process based on act.Pool, you need to embed it in your object. For example:

act.Pool uses the act.PoolBehavior interface to interact with your object:

The Init method is mandatory for implementation in act.PoolBehavior, while the other methods are optional. Similar to act.Actor, act.Pool has the embedded gen.Process interface, allowing you to use its methods directly from your object:

act.PoolOptions

With act.PoolOptions, you can configure various parameters for the pool, such as:

• PoolSize: Determines the number of worker processes to be launched when your pool process starts. If this option is not set, the default value of 3 is used.

• WorkerFactory: A factory function that creates worker processes.

• WorkerMailboxSize: Specifies the size of the mailbox for each worker process.

• WorkerArgs: Arguments passed during the startup of worker processes.

Load distribution

All asynchronous messages and synchronous requests received in the pool process's mailbox are automatically forwarded to worker processes, distributing the load evenly. The load is distributed using a FIFO queue of worker processes.

Message/Request Handling Algorithm:

1. Select a worker from the FIFO queue.

2. Forward the message/request.

3. If errors like gen.ErrProcessUnknown or gen.ErrProcessTerminated occur, a new worker is started, and the message is sent to it.

4. If gen.ErrMailboxFull occurs, the worker is requeued, and the next worker is selected.

5. Upon successful forwarding, the worker returns to the end of the queue.

If all workers are busy (e.g., due to gen.ErrMailboxFull), the pool process logs an error, such as "no available worker process. ignored message from <PID>."

Forwarding to workers only happens for messages from the Main queue. To ensure a message is handled by the pool process itself, send it with a priority higher than gen.MessagePriorityNormal. The gen.Process interface provides SendWithPriority and CallWithPriority methods for sending messages or making synchronous requests with a specified priority.

Methods of act.Pool

To dynamically manage the number of worker processes, act.Pool provides the following methods:

• AddWorkers: This method adds a specified number of worker processes to the pool. Upon successful execution, it returns the total number of worker processes in the pool after the addition.

• RemoveWorkers: This method stops and removes a specified number of worker processes from the pool. Upon successful execution, it returns the total number of worker processes remaining in the pool after the removal.

Intro

The actor model was developed in the 1970s for concurrent and parallel computations. It defines some key rules for how system components should behave and interact with each other. The main idea is that interactions between program components are not conducted through function or procedure calls but through the exchange of asynchronous messages. You can read more about the actor model and its history in the Wikipedia article.

One of the most popular programming languages that uses this model is Erlang, where the actor model is at the core of its BEAM virtual machine. In Erlang, actors are represented by lightweight processes within the virtual machine.

There are also implementations of this model in other languages. For example, in Java, the Akka framework provides a powerful tool for building solutions based on the actor model. Akka brings the principles of the actor model to the Java ecosystem, enabling developers to create highly concurrent, distributed, and fault-tolerant applications.

Actor in Ergo Framework

The Ergo Framework implements the actor model based on lightweight processes. Each process can send and receive asynchronous messages, make synchronous calls, and spawn new processes.

A process in Ergo Framework is a lightweight entity running on top of a goroutine, built around the actor model. Each process has a mailbox for incoming messages. By default, this mailbox is of unlimited size, but it can be limited by setting an appropriate parameter when the process is started. The mailbox contains four queues: Main, System, Urgent, and Log. These queues determine the priority of message processing.

An actor in Ergo Framework is an abstraction over such a process, with a set of callbacks for handling incoming messages. The standard library includes general-purpose actors like act.Actor, act.Supervisor, and act.Pool, as well as a specialized actor for handling HTTP requests, act.WebHandler.

Mailbox processing in an actor is done sequentially in a FIFO order, but with respect to priority. Messages from the Urgent queue are processed first, followed by those in the System queue. If these queues are empty, messages from the Main queue are processed. Typically, messages from the node (e.g., a request to stop the process) are placed in the Urgent and System queues. All other messages are delivered to the Main queue by default. Messages in the Log queue are processed with the lowest priority.

The additional application library for Ergo Framework contains packages with a narrow specialization or external dependencies since Ergo Framework adheres to the "zero dependencies" principle.

You can find the source code of these applications in the application library repository at https://github.com/ergo-services/application.

An extra library of meta-process implementations not included in the standard Ergo Framework library. This library contains packages with a narrow specialization. It also includes packages with external dependencies, as Ergo Framework adheres to a "zero dependency" policy.

The Application Observer provides a convenient web interface to view node status, network activity, and running processes in the node built with Ergo Framework. Additionally, it allows you to inspect the internal state of processes or meta-processes. The application is can also be used as a standalone tool Observer. For more details, see the section Inspecting With Observer. You can add the Observer application to your node during startup by including it in the node's startup options:

import (
	"ergo.services/ergo"
	"ergo.services/application/observer"
	"ergo.services/ergo/gen"
)

func main() {
	opt := gen.NodeOptions{
		Applications: []gen.ApplicationBehavior {
			observer.CreateApp(observer.Options{}),
		}
	}
	node, err := ergo.StartNode("example@localhost", opt)
	if err != nil {
		panic(err)
	}
	node.Wait()
}

The function observer.CreateApp takes observer.Options as an argument, allowing you to configure the Observer application. You can set:

• Port: The port number for the web server (default: 9911 if not specified).

• Host: The interface name (default: localhost).

• LogLevel: The logging level for the Observer application (useful for debugging). The default is gen.LogLevelInfo

These mechanisms allow processes to respond to termination events of targets with which links or monitors have been created. Such targets can include:

• A process identifier (gen.PID)

• A process name (gen.Atom or gen.ProcessID)

• A process alias (gen.Alias)

• An event (gen.Event)

• A node name (gen.Atom, in the case of monitoring a node connection)

The key difference between linking and monitoring lies in the response to the termination event of the target. When a process creates a link with a target, and that target terminates, the linked process receives an exit signal, which generally causes it to terminate as well.

In contrast, when a process creates a monitor with a target, and the target terminates, the monitoring process receives a gen.MessageDown* message, which contains the reason for the target's termination. This notification allows the monitoring process to react accordingly without necessarily terminating itself.

Links

In Ergo Framework, links are created between processes to ensure that termination events of one process propagate to linked processes. The gen.Process interface provides the following methods for creating and managing links:

• Link: This universal method allows you to create a link to a target, which can be a gen.Atom (for a local process), gen.PID, gen.ProcessID, or gen.Alias. The process will receive an exit signal upon the target process's termination. To remove the link, use the Unlink method with the same argument.

• LinkPID: Creates a link to a process by its gen.PID. The process will receive an exit signal upon the target process's termination. To remove the link, use UnlinkPID or Unlink with the same argument.

• LinkProcessID: Creates a link to a process by its gen.ProcessID. The process will receive an exit signal upon the target process's termination or when the associated name is unregistered. To remove the link, use UnlinkProcessIDor Unlink.

• LinkAlias: Creates a link to a process by its gen.Alias. The process will receive an exit signal upon the termination of the process that created the alias or upon the alias's deletion. To remove the link, use UnlinkAlias or Unlink.

• LinkNode: Creates a link to a node connection using the node name (gen.Atom). The process will receive an exit signal if the connection to the target node is lost. To remove the link, use UnlinkNode.

• LinkEvent: Creates a link to an event (gen.Event). The process will receive an exit signal when the producer process of the event terminates or when the gen.Event is unregistered. To remove the link, use UnlinkEvent. More information about this mechanism can be found in the Events section.

Exit signals are always delivered with the highest priority and placed in the Urgent queue of the process's mailbox.

Monitors

Monitors allow a process to observe the lifecycle of a target process or event. The gen.Process interface provides several methods for creating monitors:

• Monitor: A universal method for creating a monitor. The target can be a gen.Atom (for a local process), gen.PID, gen.ProcessID, or gen.Alias. The monitoring process will receive a gen.MessageDown* message upon the termination of the target process. To remove the monitor, use the Demonitor method with the same argument.

• MonitorPID: Creates a monitor for a process by its gen.PID. The monitoring process will receive a gen.MessageDownPID message upon the target process's termination, with the reason for termination included in the Reason field. To remove the monitor, use DemonitorPID or Demonitor.

• MonitorProcessID: Creates a monitor for a process by its gen.ProcessID. The monitoring process will receive a gen.MessageDownProcessID message upon the target process's termination, with the reason for termination in the Reason field. If the target process unregisters its name, the Reason field of the gen.MessageDownProcessID will contain gen.ErrUnregistered. To remove the monitor, use DemonitorProcessID or Demonitor.

• MonitorAlias: Creates a monitor for a process by its gen.Alias. The monitoring process will receive a gen.MessageDownAlias message upon the termination of the process that created the alias or if the alias is deleted. In the case of alias deletion, the Reason field of the gen.MessageDownAlias will contain gen.ErrUnregistered. To remove the monitor, use DemonitorAlias or Demonitor.

• MonitorNode: Creates a monitor for a node connection. The monitoring process will receive a gen.MessageDownNode message upon the disconnection of the node. To remove the monitor, use DemonitorNode.

• MonitorEvent: Creates a monitor for an event (gen.Event). The monitoring process will receive a gen.MessageDownEvent message when the producer of the event terminates or if the event is unregistered. If the event is unregistered, the Reason field of the gen.MessageDownEvent will contain gen.ErrUnregistered. To remove the monitor, use DemonitorEvent. More details about this mechanism can be found in the Events section.

Messages of type gen.MessageDown* are delivered with high priority and placed in the System queue of the process's mailbox.

Remote targets

Thanks to network transparency, the linking and monitoring mechanisms in Ergo Framework can be used with targets on remote nodes.

When a process creates a link or monitor with a remote target (such as a process, alias, or event), if the connection to the node where the target resides is lost, the process will receive an exit signal or a gen.MessageDown* message, depending on the mechanism used. In this case, the reason for termination will be gen.ErrNoConnection. This allows processes to handle network disconnections and remote target failures seamlessly, as part of the same fault-tolerant mechanisms used for local processes.

In Ergo Framework, an Application is a component that allows you to group a set of actors (and/or supervisors) and manage them as a single entity. The Application is responsible for starting each actor within the group and handling dependencies on other Applications if specified in the configuration. Additionally, the Application monitors the running processes and, depending on the mode chosen during startup, may stop all group members and itself if one of the processes terminates for any reason.

Create Application

To create an Application in Ergo Framework, the gen.ApplicationBehavior interface is provided:

type ApplicationBehavior interface {
	// Load invoked on loading application using method ApplicationLoad 
	// of gen.Node interface.
	Load(node Node, args ...any) (ApplicationSpec, error)
	// Start invoked once the application started
	Start(mode ApplicationMode)
	// Terminate invoked once the application stopped
	Terminate(reason error)
}

All methods of the gen.ApplicationBehavior interface must be implemented.

For example, if you want to combine an actor A1 and a supervisor S1 into a single application, the code for creating such an Application would look like this:

func createApp() gen.ApplicationBehavior {
	return &app{}
}

type app struct{}

func (a *app) Load(node gen.Node, args ...any) (gen.ApplicationSpec, error) {
	return gen.ApplicationSpec{
		Name: "my_app",
		Group: []gen.ApplicationMemberSpec{
			{
				Name:    "a1", // start with registered name
				Factory: factory_A1,
			},
			{
				// omitted field Name. will be started with no name
				Factory: factory_S1,
			},
		},
	}, nil

}
func (a *app) Start(mode gen.ApplicationMode) {} // no need? keep it empty
func (a *app) Terminate(reason error)         {}

...
myApp := createApp()
appName, err := node.ApplicationLoad(myApp)

Since an Application, by its nature, is not a process, its startup is divided into two stages: loading and starting.

To load the application, the gen.Node interface provides the method ApplicationLoad. You pass your implementation of the ApplicationBehavior interface to this method. The node will then call the Load callback method of the gen.ApplicationBehavior interface, which must return the application's specification as a gen.ApplicationSpec:

type ApplicationSpec struct {
	Name        Atom
	Description string
	Version     Version
	Depends     ApplicationDepends
	Env         map[Env]any
	Group       []ApplicationMemberSpec
	Mode        ApplicationMode
	Weight      int
	LogLevel    LogLevel
}

In the gen.ApplicationSpec structure, the Name field defines the name under which your application will be registered. This name must be unique within the node. However, the registry for application names and the registry for process names are separate. Although it is possible to run a process with a name identical to the application name, this could lead to confusion, so care should be taken when choosing names for both processes and applications. The Description field is optional.

The Group field contains a group of specifications for launching processes, defined by gen.ApplicationMemberSpec. If the Name field in gen.ApplicationMemberSpec is not empty, the process will be launched using the SpawnRegistermethod of the gen.Node interface. The processes in the group are launched sequentially, following the order specified in the Group field.

The Env field is used to set environment variables for all processes started within the application. Environment variables are inherited in the following order: Node > Application > Parent > Process.

If your application depends on other applications or network services, you can specify these dependencies using the Depends field. These dependencies are taken into account when starting the application.

The Mode field defines the startup mode for the application and its behavior if one of the processes in the group stops. This value serves as the default mode when the application is started using the ApplicationStart(...) method of the gen.Node interface. However, the startup mode can be overridden using the methods ApplicationStartTransient, ApplicationStartTemporary, or ApplicationStartPermanent of the gen.Node interface.

Finally, the LogLevel field allows you to set the logging level for the entire group of processes. If this option is not specified, the node's logging level will be used. However, individual processes can override this setting by specifying it in the ApplicationMemberSpec.Options during the process startup specification.

Application Startup Modes

Ergo Framework provides three startup modes for applications, each determining how the application behaves when one of its processes stops:

1. gen.ApplicationModeTemporary: This is the default startup mode if no specific mode is indicated in the application's specification (gen.ApplicationSpec). In this mode, the termination of any individual process (for any reason) does not lead to the shutdown of the entire application. The application will only stop when all the processes in the application group, as defined in the Group field of the gen.ApplicationSpec, have stopped. The application will always stop with the reason gen.TerminationReasonNormal.

2. gen.ApplicationModeTransient: The application stops only when one of its processes terminates unexpectedly (with a reason other than gen.TerminationReasonNormal or gen.TerminationReasonShutdown). In such cases, all running processes will receive an exit signal with the reason gen.TerminationReasonShutdown. Once all processes have stopped, the application will terminate with the reason that caused the initial process failure. If all processes complete normally, the application will stop with the reason gen.TerminationReasonNormal.

3. gen.ApplicationModePermanent: In this mode, the termination of any process in the application, for any reason, will trigger the shutdown of the entire application. All running processes will receive an exit signal with the reason gen.TerminationReasonShutdown. Once all processes have stopped, the application will terminate with the reason that caused the shutdown. If all processes complete without errors, the application will stop with the reason gen.TerminationReasonNormal.

Processes that are part of the application group can launch child processes. These child processes will inherit the application's attributes, but their termination does not affect the application's logic; their role is informational. You can determine whether a process belongs to an application using the Info method of the gen.Process interface or the ProcessInfo method of the gen.Node interface. Both methods return a gen.ProcessInfo structure, which contains the Application field indicating the process's application affiliation.

Starting an Application

In Ergo Framework, the gen.Node interface provides several methods to start an application:

• ApplicationStart: Starts the loaded application in the mode specified by gen.ApplicationSpec.Mode.

• ApplicationStartTemporary: Starts the application in gen.ApplicationModeTemporary, regardless of the mode specified in the application's specification.

• ApplicationStartTransient: Starts the application in gen.ApplicationModeTransient, regardless of the mode specified in the application's specification.

• ApplicationStartPermanent: Starts the application in gen.ApplicationModePermanent, regardless of the mode specified in the application's specification.

Before starting the application's processes, the node checks the dependencies specified in gen.ApplicationSpec.Depends. If the application depends on other applications, the node will attempt to start them first. Therefore, all dependent applications must either already be loaded or running before starting the application. If the dependencies are not met, the method will return the error gen.ErrApplicationDepends.

Once all dependencies are satisfied, the node starts launching the processes according to the order specified in gen.ApplicationSpec.Group. If any process fails to start, all previously started processes will be forcibly stopped using the Kill method of the gen.Node interface, and the startup process will return an error.

After all processes are successfully started, the Start callback method of the gen.ApplicationBehavior interface is called, and the application transitions to the running state.

You can retrieve information about the application using the ApplicationInfo method of the gen.Node interface. This method returns a gen.ApplicationInfo structure that contains summary information about the application and its current state.

Stopping an Application

To stop an application, the gen.Node interface provides the ApplicationStop method. When this method is called, the application sends an exit signal to all of its processes with the reason gen.TerminateReasonShutdown and waits for them to stop completely.

If the processes do not stop within 5 seconds, a timeout error will be returned. If the ApplicationStop method is called again while the application is still waiting for its processes to stop, it will return the error gen.ErrApplicationStopping.

For forcefully stopping the application's processes, the ApplicationStopForce method is available in the gen.Nodeinterface. In this case, all of the application's processes will be forcibly terminated using the Kill method of the gen.Nodeinterface.

Additionally, the gen.Node interface provides the ApplicationStopWithTimeout method, which allows you to specify a custom timeout period for the application to stop.

Once all of the application's processes have stopped, the node will call the Stop method of the gen.ApplicationBehavior interface, completing the shutdown process.

In Ergo Framework, a process is a lightweight entity that operates on top of a goroutine and is based on the actor model. Each process has a mailbox for receiving incoming messages. The size of the mailbox is determined by the gen.ProcessOptions.MailboxSize parameter when the process is started. By default, this value is set to zero, which makes the mailbox unlimited in size. Inside the mailbox, there are several queues—Main, System, Urgent, and Log—that prioritize the processing of messages.

Any running process can send messages and make synchronous calls to other processes, including those running on remote nodes. Additionally, a process can spawn new processes, both locally and on remote nodes (see Remote Spawn Process).

Process Identifiers

Processes in Ergo Framework are created and run within the scope of a node. Each process is assigned a unique identifier called gen.PID when it is started. The gen.PID consists of several key components:

• Node: name of the node on which the process is running.

• ID: unique number that identifies the process within its node.

• Creation: property that represents the "incarnation" of the process, meaning the specific instance of the node in which the process was created.

This combination of properties allows for:

• Identification Across Nodes: node name is used for routing messages to remote nodes where the process resides. Once routed to the correct node, the process's unique sequential number is used for local message routing on the remote node.

• Process Incarnation Handling: Creation property allows the system to distinguish between different instances of a node. This property has the same value for all processes within a node's lifecycle. If the node is restarted, the Creation value changes (it stores the timestamp of the node's startup).

If you attempt to send a message to a process with a gen.PID that belongs to a previous incarnation of the node, an error gen.ErrProcessIncarnation will be returned.

This mechanism ensures that processes are accurately identified and that message routing remains reliable, even across restarts of nodes.

gen.ProcessID

A process can also be identified by a name associated with it. You can register such a name when starting a process using the SpawnRegister method of the gen.Node interface. Additionally, a process can register its name after it has started by calling the RegisterName method of the gen.Process interface.

If you want to change the associated name of a process, you first need to unregister the current name. This is because a process is limited to having only one associated name at a time. The UnregisterName method of the gen.Processinterface allows you to remove the current name registration.

To send a message to a process by its registered name, you can use the SendProcessID method of the gen.Processinterface. For making a synchronous request by name, the gen.Process interface provides the CallProcessID method.

These methods make it easy to send messages and interact with processes using their associated names, without needing to know their gen.PID, especially when dealing with remote or dynamically created processes.

type myActor struct {
    act.Actor
}
...
func (a *myActor) HandleMessage(from gen.PID, message any) error {
    ...
    remoteProcess := gen.ProcessID{Name:"p1", Node:"node2@localhost"}
    // sending async message to the remote process with associated name
    a.SendProcessID(remoteProcess, "hello")
    // making a sync request to the remote process with associated name
    result, err := a.CallProcessID(remoteProcess, "hi")
    ...
    // you can also use methods Send and Call
    a.Send(remoteProcess)
    result, err = a.Call(remoteProcess)
    ...
}

To send a message or make a synchronous request to a local process, you can use the universal methods Send and Call of the gen.Process interface, respectively:

 func (a *myActor) HandleMessage(from gen.PID, message any) error {
    ...
    localProcess := gen.Atom("local")
    a.Send(localProcess, "hello")
    result, err := a.Call(localProcess, "hi")
    ...
 }

gen.Alias

In Ergo Framework, processes have the ability to create temporary identifiers known as aliases. A process can generate an unlimited number of aliases using the CreateAlias method of the gen.Process interface. These aliases remain valid throughout the lifetime of the process or until the process explicitly removes them using the DeleteAlias method of the gen.Process interface.

Aliases provide additional flexibility by allowing a process to be referenced by multiple temporary identifiers, useful for handling specific tasks or interactions where a unique, temporary process reference is needed.

Process states

A process in Ergo Framework can be in one of the following states:

• Init - This is the state of the process during its startup. At this stage, the process is not yet registered in the node, so only a limited set of methods from the gen.Process interface are available—such as modifying process properties (environment variables, message compression settings, setting priorities for outgoing messages). Attempts to use methods like RegisterName, RegisterEvent, CreateAlias, Call*, Link*, and Monitor* will return the error gen.ErrNotAllowed. However, methods like Send* and Spawn* remain available in this state.

• Sleep - This state occurs when the process has no messages in its mailbox, and no goroutine is running for the process. The process is idle and consumes no CPU resources in this state.

• Running - When a process receives a message in its mailbox, a goroutine is launched, and the appropriate callback is executed. Once the message has been processed and if there are no other messages in the mailbox, the process returns to the Sleep state, and the goroutine is terminated.

• WaitResponse - The process transitions to this state from Running when it makes a synchronous request (Call*). The process waits for a response to the request, and once received, it transitions back to the Running state

• Zombee - This is an intermediate state that occurs when a process was in the Running state but the node forcibly stopped the process using the Kill method. In this state, the process cannot access any methods from the gen.Process interface. After processing the current message and terminating the goroutine, the process transitions from Zombie to Terminated. In this state, the process no longer receives new messages.

• Terminated - This is the final state of the process before it is removed from the node. Messages are not delivered to a process in this state.

You can retrieve the current state of a process using the ProcessInfo method of the gen.Node interface. Alternatively, you can use the State method of the gen.Process interface to check the state directly from the process.

These states define the lifecycle of a process and determine what actions it can perform at any given moment in Ergo Framework.

Environment Variables

Each process in Ergo Framework has its own set of environment variables, which can be retrieved using the EnvList()method of the gen.Process interface. To modify these variables, the SetEnv(name gen.Env, value any) method is used; if value is set to nil, the environment variable with the specified name will be removed. Environment variable names are case-insensitive, allowing for flexible and dynamic configuration management within processes:

type myActor struct {
    act.Actor
}

func (a *myActor) Init(args ...any) error {
    ...
    a.SetEnv("var", 1) // set variable
    a.SetEnv("Var", 2) // overwrite value, the name is case insensitive
    v, _ := a.Env("vAr") // return 2 as a value
    a.SetEnv("vaR", nil) // remove variable
    ...
}

The initialization of a process's environment variables occurs at the moment the process starts. The starting process first inherits the environment variables of the node, then the environment variables of the application and parent process, and finally, the environment variables specified in gen.ProcessOptions are added.

Process starting

To launch a new process in Ergo Framework, there are two methods:

Spawn(factory ProcessFactory, options ProcessOptions, args ...any) (PID, error)
SpawnRegister(register Atom, factory ProcessFactory, options ProcessOptions, args ...any) (PID, error)

Both methods are available in the gen.Node and gen.Process interfaces.

When using the SpawnRegister method, the node checks whether the specified name register can be assigned to the starting process. If the name is already taken by another process, this function will return the error gen.ErrTaken.

Parent process

Processes launched by the node (using the Spawn* methods of the gen.Node interface) are assigned a virtual parent process identifier, known as CorePID. You can retrieve this value using the CorePID method of the gen.Node interface.

If a process is launched using the Spawn* methods of the gen.Process interface, it becomes a child process. The Parentmethod of the gen.Process interface in the child process will return the identifier of the process that launched it.

Attempting to send an exit signal to a parent process using the SendExit method of the gen.Process interface will result in the error gen.ErrNotAllowed.

Additionally, the parent process identifier is used in handling exit signals in act.Actor (see the section on TrapExit in Actor).

Process leader

Each process in Ergo Framework has a process leader identifier, in addition to the parent process identifier. You can retrieve this identifier using the Leader method of the gen.Process interface. When a process is started by the node, its process leader is set to the virtual identifier CorePID. If the process is started by another process, the child process inherits the leader identifier from its parent process.

During process startup, it is also possible to explicitly set a process leader using the Leader option in gen.ProcessOption. This is commonly done when starting child processes in act.Supervisor. While the process leader identifier does not influence the operational logic of processes, it serves an informational role and can be used within your application's logic as needed.

gen.ProcessFactory

To start a process in Ergo Framework, you need to specify a factory argument, which is a factory function. This function must return an object that implements the gen.ProcessBehavior interface. This is a low-level interface that is already implemented in general-purpose actors (such as act.Actor, act.Supervisor, etc.). Therefore, in most cases, you only need to embed the required actor into your object to satisfy the gen.ProcessBehavior interface.

Example:

// based on act.Actor
type MyActor struct {
    act.Actor // embed generic actor
}
func factoryMyActor() gen.ProcessBehavior {
    return &MyActor{}
}
...
node.Spawn(factoryMyActor, gen.ProcessOptions{})

gen.ProcessOptions

The options parameter in the Spawn or SpawnRegister functions allows you to configure various parameters for the process being launched:

• MailboxSize: defines the size of the mailbox. If not specified, the mailbox will be unlimited in size.

• Leader: sets the leader of the process group. If not specified, the leader value is inherited from the parent process. This setting is informational and does not affect the process itself but can be used in your application's logic. Typically, this is set by supervisor processes.

• Compression: specifies the message compression settings when sending messages over the network to remote nodes. This option is ignored if the recipient is a local process.

• SendPriority: allows you to set the priority for sent messages. There are three priority levels:

  • gen.MessagePriorityNormal: Messages are delivered to the Main queue of the recipient's mailbox.

  • gen.MessagePriorityHigh: Messages are delivered to the System queue.

  • gen.MessagePriorityMax: Highest-priority messages are delivered to the Urgent queue of the recipient's mailbox.

• ImportantDelivery: enables the important flag for messages sent to remote processes. When enabled, the remote node will send an acknowledgment that the message was delivered to the recipient's mailbox. If the message cannot be delivered, an error (e.g., gen.ErrProcessTerminated, gen.ProcessUnknown, gen.ProcessMailboxFull) will be returned. You can also manage this flag using the SetImportantDelivery method of the gen.Process interface. This flag only applies to messages sent via the SendPID, SendProcessID, and SendAlias methods or requests made using CallPID, CallProcessID, or CallAlias methods of gen.Process. To force a message to be sent with the important flag, use the SendImportant or CallImportant methods of gen.Process.

• Fallback: specifies a fallback process to which messages will be redirected in the event that the recipient's mailbox is full. Each such message is wrapped in the gen.MessageFallback structure. This option is ignored if the recipient process's mailbox is unlimited.

• LinkParent: Automatically creates a link with the parent process when the process is started (after successful initialization). This option is ignored if the process is started by the node.

• LinkChild: Automatically creates a link with a child process upon its successful startup (after successful initialization). This option is ignored if the process is started by the node.

• Env: Sets the environment variables for the process.

Process termination

In Ergo Framework, a process can be terminated in the following ways:

1. Forcefully: process can be terminated by force using the Kill method provided by the gen.Node interface.

2. Sending an Exit Signal: you can stop a process by sending it an exit signal with a specified reason. This is done using the SendExit method, which is available in both the gen.Node and gen.Process interfaces. The exit signal is always delivered with the highest priority and placed in the Urgent queue. In act.Actor, there is a mechanism to intercept exit signals, allowing them to be handled like normal gen.MessageExit* messages. More information on this mechanism can be found in the Actor section. Sending an exit signal to yourself, your parent process, or the process leader is not allowed. In such cases, the SendExit method will return the error gen.ErrNotAllowed.

3. Self-Termination: A process can terminate itself by returning a non-nil value (of type error) from its message handling callback. The returned value will be treated as the termination reason, and the process will be terminated. You can use gen.TerminateReasonNormal or gen.TerminateReasonShutdown for normal shutdown scenarios.

4. Panic: If a panic occurs during message processing, the process will be terminated with the reason gen.TerminateReasonPanic.

Upon termination, all resources associated with the process are released:

• All events registered by the process are unregistered.

• Any associated name, if registered, is freed.

• Aliases, links, and monitors created by the process are removed.

• The process is removed from the logging system if it was previously registered as a logger.

• All meta-processes launched by the process are terminated.

Process information

You can retrieve summary information about a process using the ProcessInfo method of the gen.Nodeinterface or the Info method of the gen.Process interface. Both methods return a gen.ProcessInfo structure containing detailed information about the process.

An extra library of registrars or client implementations not included in the standard Ergo Framework library. This library contains packages with a narrow specialization. It also includes packages with external dependencies, as Ergo Framework follows a "zero dependency" policy

This package implements functionality for working with WebSocket connections through two meta-processes: WebSocket Handler (for handling HTTP requests to create WebSocket connections) and WebSocket Connection (for managing the established WebSocket connection). This implementation depends on the external library https://github.com/gorilla/websocket, which is why it is placed in a separate package "ergo.services/meta/websocket". You can find the source code in the additional meta-process library repository at https://github.com/ergo-services/meta.

WebSocket Handler

To handle incoming WebSocket connections, use the CreateHandler function. This function returns an object implementing gen.MetaProcess and http.Handler, needed for launching the meta-process and processing HTTP requests. It accepts websocket.HandlerOptions as arguments:

• ProcessPool: Specifies a list of process names to handle WebSocket messages.

• HandshakeTimeout: Limits the time for the upgrade process.

• EnableCompression: Enables compression for WebSocket connections.

• CheckOrigin: Allows setting a function to verify the request's origin.

After creation, start the meta-process with SpawnMeta. Each connection launches a new WebSocket Connection meta-process for handling.

WebSocket Connection

This package implements the functionality of a meta-process for handling a WebSocket connection. It also allows initiating a WebSocket connection. To initiate a connection, use the CreateConnection function, which takes websocket.ConnectionOptions as an argument:

type ConnectionOptions struct {
	Process           gen.Atom
	URL               url.URL
	HandshakeTimeout  time.Duration
	EnableCompression bool
}

This structure is similar to websocket.HandlerOptions but includes an additional field, URL, where the WebSocket server address must be specified. For example:

opt := websocket.ClientOptions{
	URL: url.URL{Scheme: "ws", Host: "localhost:8080", Path: "/ws"},
}

During the execution of the CreateConnection function, it attempts to establish a WebSocket connection with the specified server.

After creating the meta-process, you must launch it using the SpawnMeta method from the gen.Process interface. If an error occurs while launching the meta-process, you need to call the Terminate method from the gen.MetaBehavior interface to close the WebSocket connection created by CreateConnection:

func(a *MyActor) HandleMessage(from gen.PID, message any) error {
    // ...
    opt := websocket.ConnectionOptions{
	URL: url.URL{Scheme: "ws", Host: "localhost:8080", Path: "/ws"},
    }
    wsconn, err := websocket.CreateConnection(opt)
    if err != nil {
        a.Log().Error("unable to connect to %s: %s", opt.URL.String(), err)
        return nil
    }
    id, err := a.SpawnMeta(wsconn, gen.MetaOptions{})
    if err != nil {
        // unable to spawn meta-process. need to close the connection
        wsconn.Terminate(err)
        a.Log().Error("unable to spawn meta-process: %s", err)
        return nil
    }
    a.Log().Info("spawned meta-process %s to handle connection with %s", id, opt.URL.String())
    // ...
    return nil
}

WebSocket messages

When working with WebSocket connections, three types of messages are used:

• websocket.MessageConnect - Sent to the process when the meta-process handling the WebSocket connection is started:

  Copy

  type MessageConnect struct {
  	ID         gen.Alias
  	RemoteAddr net.Addr
  	LocalAddr  net.Addr
  }

  The ID field in websocket.MessageConnect contains the identifier of the meta-process handling the WebSocket connection

• websocket.MessageDisconnect - is sent to the process when a WebSocket connection is disconnected:

  Copy

  type MessageDisconnect struct {
  	ID         gen.Alias
  }

  The ID field in websocket.MessageDisconnect contains the identifier of the meta-process that handled the WebSocket connection. After the connection is terminated, the meta-process finishes its work and sends this message during its termination phase, signaling that the WebSocket connection has been closed

• websocket.Message - is used for both receiving and sending WebSocket messages:

  Copy

  type Message struct {
  	ID   gen.Alias
  	Type MessageType
  	Body []byte
  }

  ID - identifier of the meta-process handling the WebSocket connection. Body - the message's body. Type - type of WebSocket message, according to the WebSocket specification. Several message types are available:

  Copy

  const (
  	MessageTypeText   MessageType = 1
  	MessageTypeBinary MessageType = 2
  	MessageTypeClose  MessageType = 8
  	MessageTypePing   MessageType = 9
  	MessageTypePong   MessageType = 10
  )

If the meta-process fails to send a message to the process, the WebSocket connection is terminated, and the meta-process itself also terminates.

Sending a WebSocket-message

To send a message over a WebSocket connection, use the Send (or SendAlias) method from the gen.Process interface. The recipient should be the gen.Alias of the meta-process handling the WebSocket connection.

The message must be of type websocket.Message. If the Type field is not explicitly set, the default is websocket.MessageTypeText. The websocket.Message.ID field is ignored when sending (it's used only for incoming messages).

For a complete implementation of a WebSocket server, see the project in the repository https://github.com/ergo-services/examples, the websocket project:

This package implements the gen.LoggerBehavior interface and provides the ability to output log messages to standard output with color highlighting.

Below is a demonstration of log messages (from nodes, processes, and meta-processes) with different logging levels:

Format

<time> <level> <log source> [process name] [process behavior]: <log message>

When logging, the package also highlights in color the types gen.Atom, gen.PID, gen.ProcessID, gen.Ref, gen.Alias, gen.Event.

Available options

• ShortLevelName Displays the shortened name of the log level

• IncludeBehavior Includes the name of the process behavior in the log message

• IncludeName includes the registered name of the process in the log message

Example

To create a client, use the Create function from the saturn package. The function requires:

• The hostname where the central registrar is running (default port: 4499, unless specified in saturn.Options)

• A token for connecting to Saturn

• a set of options saturn.Options

Then, set this client in the gen.NetworkOption.Registrar options

Using saturn.Options, you can specify:

• Cluster - The cluster name for your node

• Port - The port number for the central Saturn registrar

• KeepAlive - The keep-alive parameter for the TCP connection with Saturn

• InsecureSkipVerify - Option to ignore TLS certificate verification

Additionally, this library registers a gen.Event and generates messages based on events received from the central Saturn registrar within the specified cluster. This allows the node to stay informed of any updates or changes within the cluster, ensuring real-time event-driven communication and responsiveness to cluster configurations:

• saturn.EventNodeJoined - Triggered when another node is registered in the same cluster.

• saturn.EventNodeLeft - Triggered when a node disconnects from the central registrar

• saturn.EventApplicationLoaded - An application was loaded on a remote node. Use ResolveApplication from the gen.Resolver interface to get application details

• saturn.EventApplicationStarted - Triggered when an application starts on a remote node.

• saturn.EventApplicationStopping - Triggered when an application begins stopping on a remote node.

• satrun.EventApplicationStopped - Triggered when an application is stopped on a remote node.

• saturn.EventApplicationUnloaded - Triggered when an application is unloaded on a remote node

• saturn.EventConfigUpdate - The node's configuration was updated

To receive such messages, you need to subscribe to Saturn client events using the LinkEvent or MonitorEvent methods from the gen.Process interface. You can obtain the name of the registered event using the Event method from the gen.Registrar interface. This allows your node to listen for important cluster events like node joins, application starts, configuration updates, and more, ensuring real-time updates and handling of cluster changes.

You can also use the Config and ConfigItem methods from the gen.Registrar interface to retrieve configuration parameters from the registrar.

To get information about available applications in the cluster, use the ResolveApplication method from the gen.Resolver interface, which returns a list of gen.ApplicationRoute structures:

• Name The name of the application

• Node The name of the node where the application is loaded or running

• Weight The weight assigned to the application in gen.ApplicationSpec

• Mode The application's startup mode (gen.ApplicationModeTemporary, gen.ApplicationModePermanent, gen.ApplicationModeTransient)..

• State The current state of the application (gen.ApplicationStateLoaded, gen.ApplicationStateRunning, gen.ApplicationStateStopping)

You can access the gen.Resolver interface using the Resolver method from the gen.Registrar interface.

The network stack in Ergo Framework includes the capability to spawn processes on remote nodes. Nodes can control access to this feature or disable it entirely by using the EnableRemoteSpawn flag in gen.NetworkFlags. By default, this feature is enabled.

Access Control

To manage the ability of remote nodes to spawn processes, the gen.Network interface provides two methods:

EnableSpawn(name gen.Atom, factory gen.ProcessFactory, nodes ...gen.Atom) error
DisableSpawn(name gen.Atom, nodes ...gen.Atom) error

The name argument specifies the name under which the process factory will be registered in the network stack. Remote nodes must use this name when requesting to spawn a process.

Using EnableSpawn with the nodes argument creates an access list, allowing only the specified remote nodes to spawn processes with the registered factory. Conversely, using DisableSpawn with the nodes argument removes the specified nodes from the access list. If the list becomes empty, access is open to all nodes. To fully disable access to the process factory, use DisableSpawn without the nodes argument.

Spawn

To spawn a process on a remote node, you need to use the gen.RemoteNode interface, which can be obtained using the GetNode or Node methods of the gen.Network interface. The gen.RemoteNode interface provides two methods:

Spawn(name gen.Atom, options gen.ProcessOptions, args ...any) (gen.PID, error)
SpawnRegister(register gen.Atom, name gen.Atom, options gen.ProcessOptions, args ...any) (PID, error)

For the name argument, you must use the name under which the process factory is registered on the remote node.

Just like with local process spawning, the process started on the remote node will inherit certain parameters from its parent. In this case, the parent is the virtual identifier of the node that sent the spawn request. The process will also inherit the logging level.

To inherit environment variables, you need to enable the ExposeEnvRemoteSpawn option in gen.NodeOptions.Securitywhen starting your node. The environment variable values must be encodable and decodable using the EDF format. If you are using a custom type as the value of an environment variable, that type must be registered (see the Network Transparency section).

Upon success, these methods return the process identifier (gen.PID) of the process that was successfully started on the remote node.

You can also spawn a process on a remote node using the methods provided by the gen.Process interface:

RemoteSpawn(node gen.Atom, name gen.Atom, options gen.ProcessOptions, args ...any) (gen.PID, error)
RemoteSpawnRegister(node gen.Atom, name gen.Atom, register gen.Atom, options gen.ProcessOptions, args ...any) (gen.PID, error)

When using these methods, the process started on the remote node will inherit parameters from the parent process, including the application name, logging level, and environment variables (if the ExposeEnvRemoteSpawn flag in gen.NodeOptions.Security was enabled).

Note that linking options in gen.ProcessOptions will be ignored when spawning processes remotely.

In addition to the ability to spawn processes on remote nodes, Ergo Framework also allows you to start applications remotely. Similar to process spawning, access to this functionality is controlled by the EnableRemoteApplicationStartflag in gen.NetworkFlags. By default, this flag is enabled.

Access Control

To allow an application to be started from a remote node, it must be registered in the network stack. The gen.Networkinterface provides the following methods for controlling this functionality:

EnableApplicationStart(name gen.Atom, nodes ...gen.Atom) error
DisableApplicationStart(name gen.Atom, nodes ...gen.Atom) error

The name argument represents the application's name, and the nodes argument controls which nodes are permitted to start the application remotely. This gives you fine-grained control over which nodes can initiate the remote startup of specific applications.

Using the EnableApplicationStart method with the nodes argument creates an access list of remote nodes that are permitted to start the specified application.

When using the DisableApplicationStart method with the nodes argument, it removes the specified nodes from the access list. If the access list becomes empty after removing nodes, access is opened to all remote nodes. To fully disable access to starting the application, use the DisableApplicationStart method without the nodes argument.

Start Application

To start an application on a remote node, use the ApplicationStart method of the gen.RemoteNode interface (which you can access via the GetNode or Node methods of the gen.Network interface). This interface also provides additional methods to control the application's startup mode:

ApplicationStartTemporary(name gen.Atom, options gen.ApplicationOptions) error
ApplicationStartTransient(name gen.Atom, options gen.ApplicationOptions) error
ApplicationStartPermanent(name gen.Atom, options gen.ApplicationOptions) error

When an application is started remotely, the Parent property of the application is assigned the name of the node that initiated the startup. You can retrieve information about the application and its status using the ApplicationInfo method of the gen.Node interface.

When creating an outgoing connection to a remote node, the node first checks the internal Static Routing Table for an existing route. If no static route is found for the specified remote node's name, the node uses the Registrar and sends a resolve request to obtain the necessary connection parameters (see the Service Discovery section for more information).

To define a static route for a specific node or group of nodes, use the AddRoute method of the gen.Network interface. This method allows you to manually configure the connection parameters for outgoing connections, ensuring more control over how the node establishes connections with the specified nodes, bypassing the need to query the Registrar.

AddRoute(match string, route gen.NetworkRoute, weight int) error

• match: defines the name of the remote node or a pattern to match multiple nodes. The node uses the MatchStringmethod from Go's standard library regexp package to match the node names based on this pattern.

• route: specifies the connection parameters to be used when establishing a connection to the matched node(s). This includes details such as the network protocol, port, TLS settings, and other connection options.

• weight: defines the weight of the specified route. If multiple routes match the same node name in the routing table, the node will return a list of routes sorted by descending weight. The node will then use the route with the highest weight to establish the connection.

To check for the existence of a static route for a specific remote node name, use the Route method of the gen.Networkinterface. To remove a static route, use the RemoveRoute method of the gen.Network interface, specifying the match value that was used when adding the route.

gen.NetworkRoute

The route argument in the AddRoute method allows you to specify detailed connection parameters for establishing outgoing connections to remote nodes. This enables fine-grained control over how connections are created:

• Resolver: specifies a particular Registrar to be used by the node for resolving the connection. This interface is obtained via the Resolver method of the gen.Registrar interface. This functionality is useful when a node is working with multiple network stacks. For example, see the Erlang section for managing clusters with different network stacks simultaneously.

• Route: allows you to explicitly define the host name, port number, TLS mode, handshake version, and protocol version. If the Resolver parameter is explicitly set, the route parameters provided will override any route information returned by the Registrar.

• Cookie: overrides the default Cookie value specified in gen.NetworkOptions for the given route, providing custom authentication or security settings specific to the connection.

• Cert: specifies a CertManager to establish a TLS connection, ensuring that the appropriate certificates are used during the connection.

• Flags: you can override specific flags for the given route. For example, you can explicitly disable certain mechanisms, such as preventing the remote node from spawning processes over the established connection.

• AtomMapping: enables automatic substitution of gen.Atom values transmitted or received during the connection.

• LogLevel: defines the logging level for the network stack within the established connection. This provides granular control over the verbosity of log messages for troubleshooting or monitoring network activity on a per-connection basis

The Ergo Framework's network stack implements network transparency using three components:

Registrar - Manages node registration, including applications and additional parameters to help nodes find each other and establish connections. See the Service Discovering section for more details.

EDF (Ergo Data Format) - A data format for network communication that automatically encodes and decodes messages when sent to remote nodes.

ENP (Ergo Network Protocol) - A protocol for network interaction, facilitating asynchronous message exchange, synchronous requests, and remote process/application initiation between nodes.

Network Options

Network stack parameters can be specified at node startup in gen.NodeOptions.Network. This field, of type gen.NetworkOptions, allows you to configure the following:

• Mode: default is gen.NetworkModeEnabled. Use gen.NetworkModeHidden to disable incoming connections, while still allowing outgoing ones. Use gen.NetworkModeDisabled to completely disable networking.

• Cookie: sets a secret password for access control, used for both incoming and outgoing connections.

• MaxMessageSize: defines the maximum size of network messages.

• Flags: specify features available to remote nodes, like process or application launching.

• Registrar: by default, the node uses the built-in implementation (ergo.services/ergo/net/registrar). For central registrar Saturn or for Erlang clusters, use respective libraries (Saturn or EPMD).

• Handshake: connection setup starts with remote node authentication (cookie verification) and protocol parameter exchange. Default uses the built-in implementation (ergo.services/ergo/net/handshake).

• Proto: default protocol is Ergo Framework's ENP (ergo.services/ergo/net/proto), but other protocols like the Erlang stack's DIST can be used.

• Acceptors: defines a set of acceptors for the node ([]gen.AcceptorOptions). Each acceptor can configure port, TLS encryption, message size limits, and a custom cookie different from gen.NetworkOptions.Cookie. You can also configure different Registrar/Handshake/Proto sets for each acceptor, enabling simultaneous use of multiple network stacks, such as Ergo Framework and Erlang.

Establishing a connection with a remote node

To connect to a remote node, simply send a message to a process running on that node. The node will automatically attempt to establish a network connection before sending the message.

You can also establish a connection explicitly by calling the GetNode method from the gen.Network interface. For example:

func main() {
    ...
    node, err := ergo.StartNode("node1@localhost", gen.NodeOptions{})
    ...
    remote, err := node.Network().GetNode("node2@localhost")
    ...
}

The GetNode method returns the gen.RemoteNode interface, which provides information about the remote node. Additionally, it allows you to launch a process or application on that remote node:

type RemoteNode interface {
	Name() Atom
	Uptime() int64
	ConnectionUptime() int64
	Version() Version
	Info() RemoteNodeInfo

	Spawn(name Atom, options ProcessOptions, args ...any) (PID, error)
	SpawnRegister(register Atom, name Atom, options ProcessOptions, args ...any) (PID, error)

	// ApplicationStart starts application on the remote node.
	// Starting mode is according to the defined in the gen.ApplicationSpec.Mode
	ApplicationStart(name Atom, options ApplicationOptions) error

	// ApplicationStartTemporary starts application on the remote node in temporary mode
	// overriding the value of gen.ApplicationSpec.Mode
	ApplicationStartTemporary(name Atom, options ApplicationOptions) error

	// ApplicationStartTransient starts application on the remote node in transient mode
	// overriding the value of gen.ApplicationSpec.Mode
	ApplicationStartTransient(name Atom, options ApplicationOptions) error

	// ApplicationStartPermanent starts application on the remote node in permanent mode
	// overriding the value of gen.ApplicationSpec.Mode
	ApplicationStartPermanent(name Atom, options ApplicationOptions) error
	
	Creation() int64
	Disconnect()
}

For more details about remote process and application spawning, refer to the Remote Spawn Process and Remote Start Application sections.

You can also use the Node method from the gen.Network interface. It returns a gen.RemoteNode if a connection to the remote node exists; otherwise, it returns gen.ErrNoConnection.

To establish a connection with a remote node using specific parameters, use the GetNodeWithRoute method from the gen.Network interface. This method allows for more customized network connection settings.

Network Stack Interfaces

This section explains the internal mechanisms by which a node interacts with the network stack. Understanding these can help you implement a custom network stack or components, such as a modified version of gen.NetworkHandshake for customized authentication.

For incoming connections, the node starts acceptors with specified parameters. For outgoing connections, it independently creates TCP connections using parameters from the registrar (see Service Discovering). After establishing the TCP connection, the node leverages several interfaces to operate the network stack:

gen.NetworkHandshake

At the first stage, the node works with the gen.NetworkHandshake interface. If a TCP connection was created by an acceptor, the Accept method is called, initiating the remote node's authentication and the exchange of parameters for the network protocol. If the TCP connection was initiated by the node itself, the Start method is used, which handles the authentication process and the exchange of network protocol parameters.

The Join method is used to combine multiple TCP connections. In the ENP protocol, by default, 3 TCP connections are created and merged into one virtual connection between nodes, which improves network protocol performance. You can set the size of the TCP connection pool using the PoolSize parameter in handshake.Options (ergo.services/ergo/net/handshake). The Erlang network stack's DIST protocol does not support this feature.

type NetworkHandshake interface {
	NetworkFlags() NetworkFlags
	// Start initiates handshake process.
	Start(NodeHandshake, net.Conn, HandshakeOptions) (HandshakeResult, error)
	// Join attempts to join new TCP-connection to the existing connection 
	// to remote node
	Join(NodeHandshake, net.Conn, id string, HandshakeOptions) ([]byte, error)
	// Accept initiate handshake process for the connection created by remote node.
	Accept(NodeHandshake, net.Conn, HandshakeOptions) (HandshakeResult, error)
	// Version
	Version() Version
}

gen.NetworkProto

After successfully completing the handshake procedure, the node transfers control of the TCP connection to the network protocol using the gen.NetworkProto interface. This occurs in two stages: first, the node calls the NewConnectionmethod, which returns the gen.Connection interface for the created connection. The node then registers this connection in its internal routing mechanism with the remote node's name. If a connection with the same name already exists, the node automatically closes the TCP connection, as only one connection can exist between two nodes.

Upon successful registration, the node calls the Serve method. The completion of this method indicates the termination of the connection with the remote node, effectively closing the connection.

type NetworkProto interface {
	// NewConnection
	NewConnection(core Core, result HandshakeResult, log Log) (Connection, error)
	// Serve connection. Argument dial is the closure to create TCP connection with invoking
	// NetworkHandshake.Join inside to shortcut the handshake process
	Serve(conn Connection, dial NetworkDial) error
	// Version
	Version() Version
}

gen.Connection

The gen.Connection interface is used by the node for routing messages and requests to the remote node with which a connection has been established. If you are developing your own protocol, you will need to implement all the methods of this interface to handle communication between nodes.

type Connection interface {
	Node() RemoteNode

	// Methods for sending async message to the remote process
	SendPID(from PID, to PID, options MessageOptions, message any) error
	SendProcessID(from PID, to ProcessID, options MessageOptions, message any) error
	SendAlias(from PID, to Alias, options MessageOptions, message any) error

	SendEvent(from PID, options MessageOptions, message MessageEvent) error
	SendExit(from PID, to PID, reason error) error
	SendResponse(from PID, to PID, ref Ref, options MessageOptions, response any) error

	// target terminated
	SendTerminatePID(target PID, reason error) error
	SendTerminateProcessID(target ProcessID, reason error) error
	SendTerminateAlias(target Alias, reason error) error
	SendTerminateEvent(target Event, reason error) error

	// Methods for sending sync request to the remote process
	CallPID(ref Ref, from PID, to PID, options MessageOptions, message any) error
	CallProcessID(ref Ref, from PID, to ProcessID, options MessageOptions, message any) error
	CallAlias(ref Ref, from PID, to Alias, options MessageOptions, message any) error

	// Links
	LinkPID(pid PID, target PID) error
	UnlinkPID(pid PID, target PID) error

	LinkProcessID(pid PID, target ProcessID) error
	UnlinkProcessID(pid PID, target ProcessID) error

	LinkAlias(pid PID, target Alias) error
	UnlinkAlias(pid PID, target Alias) error

	LinkEvent(pid PID, target Event) ([]MessageEvent, error)
	UnlinkEvent(pid PID, targer Event) error

	// Monitors
	MonitorPID(pid PID, target PID) error
	DemonitorPID(pid PID, target PID) error

	MonitorProcessID(pid PID, target ProcessID) error
	DemonitorProcessID(pid PID, target ProcessID) error

	MonitorAlias(pid PID, target Alias) error
	DemonitorAlias(pid PID, target Alias) error

	MonitorEvent(pid PID, target Event) ([]MessageEvent, error)
	DemonitorEvent(pid PID, targer Event) error

	RemoteSpawn(name Atom, options ProcessOptionsExtra) (PID, error)

	Join(c net.Conn, id string, dial NetworkDial, tail []byte) error
	Terminate(reason error)
}

Cron functionality is provided to enable periodic job execution in a node. Its implementation replicates the functionality of the Cron service in Unix systems and supports the Crontab specification format.

To set up jobs at the node startup, use the gen.NodeOptions.Cron parameters. Each job is described using the gen.CronJob:

type CronJob struct {
	// Name job name
	Name gen.Atom
	// Spec time spec in "crontab" format
	Spec string
	// Location defines timezone
	Location *time.Location
	// Action
	Action gen.CronAction
	// Fallback
	Fallback gen.ProcessFallback
}

Parameters:

• Name defines the name of the job. It must be unique among all cron jobs

• Spec sets the scheduling parameters for the job in crontab format

• Location allows to set the time zone for the job's scheduling parameters. By default, the local time zone is used

• Action defines what needs to be run

• Fallback allows specifying a process to notify if the Action results in an error

The Action field has the interface type gen.CronAction. You can use the following ready-to-use implementations:

• gen.CreateCronActionMessage(to any, priority gen.MessagePriority) – creates an Action that sends a gen.MessageCron message to the specified process to. The to argument can be one of the following: gen.Atom, gen.ProcessID, gen.PID, or gen.Alias, referring to either a local process or a process on a remote node.

• gen.CreateCronActionSpawn(factory gen.ProcessFactory, options gen.CronActionSpawnOptions) – spawns a local process. The options argument specifies the startup parameters for the process.

• gen.CreateCronActionRemoteSpawn(node gen.Atom, name gen.Atom, options gen.CronActionSpawnOptions) – spawns a process on a remote node. The mechanism for spawning processes on a remote node is described in the Remote Spawn Process section.

When using CreateCronActionSpawn or CreateCronActionRemoteSpawn, the following environment variables will be added to the spawned processes:

• gen.CronEnvNodeName

• gen.CronEnvJobName

• gen.CronEnvJobActionTime

Example

In the example below, every day at 21:07 (Shanghai time), a gen.MessageCron message will be sent to the local process registered as myweb1. If the message cannot be delivered, a gen.MessageCronFallback message will be sent to the local process named myweb2:

func main() {
	var options gen.NodeOptions
	// ...
	locationAsiaShanghai, _ := time.LoadLocation("Asia/Shanghai")
	options.Cron.Jobs = []gen.CronJob{
		gen.CronJob{Name: "job1",
			Spec:     "7 21 * * *",
			Action:   gen.CreateCronActionMessage(gen.Atom("myweb1"), 
							gen.MessagePriorityNormal),
			Location: locationAsiaShanghai,
			Fallback: gen.ProcessFallback{Enable: true, Name: "myweb2"},
		},
	}
	node, err := ergo.StartNode("cron@localhost", options)
	if err != nil {
		panic(err)
	}
	// ...
}

gen.Cron interface

You can also manage jobs using the gen.Cron interface. It provides the following methods for this:

type Cron interface {
	// AddJob adds a new job
	AddJob(job gen.CronJob) error
	// RemoveJob removes new job
	RemoveJob(name gen.Atom) error
	// EnableJob allows you to enable previously disabled job
	EnableJob(name gen.Atom) error
	// DisableJob disables job
	DisableJob(name gen.Atom) error

	// Info returns information about the jobs, spool of jobs for the next run
	Info() gen.CronInfo
	// JobInfo returns information for the given job
	JobInfo(name gen.Atom) (gen.CronJobInfo, error)

	// Schedule returns a list of jobs planned to be run for the given period
	Schedule(since time.Time, duration time.Duration) []gen.CronSchedule
	// JobSchedule returns a list of scheduled run times for the given job and period.
	JobSchedule(job Atom, since time.Time, duration time.Duration) ([]time.Time, error)
}

Access to the gen.Cron interface can be obtained using the Cron method of the gen.Node interface.

Custom Action

You can also create your own Action. To do this, simply implement the gen.CronAction interface:

type CronAction interface {
	Do(job gen.Atom, node gen.Node, action_time time.Time) error
	Info() string
}

It is worth noting that the action_time argument is passed in the time zone of the job, i.e., the one specified in the gen.CronJob.Location field when the job was created.

The Info method of the interface is used when calling gen.Cron.Info() to retrieve summary information about Cron and its jobs.

Cron specification

* * * * *
| | | | |                                                 allowed format
| | | | day-of-week (1–7) (Monday to Sunday)      *   d   d,d   d-d    dL    d#d
| | | month (1–12)                                *   d   d,d   d-d   */d
| | day-of-month (1–31)                           *   d   d,d   d-d   */d   d-d/d   L
| hour (0–23)                                     *   d   d,d   d-d   */d   d-d/d
minute (0–59)                                     *   d   d,d   d-d   */d   d-d/d

• * represents "all". For example, using * * * * * will run every minute. Using * * * * 1 will run every minute only on Monday.

• - allows specifying a range of values. For example, 15 23 * * 1-3 will trigger the job from Monday to Wednesday at 23:15

• , defines a sequence of values: 15,25,35 23 * * * – this will trigger the job every day at 23:15, 23:25, and 23:35

• / used for step values: */5 3 * * * this will trigger the job every 5 minutes during the hour starting from 3:00 (3:00, 3:05 ... 3:55). It can also be used with ranges: 21-37/5 17 * * * - every day at 17:21, 17:26, 17:31 and 17:36

• # available for use only in the day-of-week field in the format day-of-week#occurrence. It allows specifying constructs like 5#2, which refers to the second Friday of the month

• L stands for "last". In the day-of-month field, it specifies the last day of the month. In the day-of-week field - allows specifying constructs such as "the last Friday" (5L) of a given month.

You can also use the following macro definitions:

• @hourly - 1 * * * * every hour

• @daily - 10 3 * * * every day at 3:10

• @monthly - 20 4 1 * * on day 1 of the month at 4:20

• @weekly - 30 5 * * 1 on Monday at 5:30

Examples

1 19 * * 1#1,7L run at 19:01 every month on the first Monday and last Sunday

15 15 10-15/3 * * run at 15:15 every month on the 10th and 13th

To view the schedule of your job, use the JobSchedule method of the gen.Cron interface. To view the schedule of all jobs, use the Schedule method of this interface.

DST (daylight saving time) and Time-adjustment support

This implementation takes time changes into account – if a time adjustment occurs at the time of the job's scheduled execution and the new time does not match the job's schedule, the job will not be executed.

For example, if a job has the specification 0 2 * * * (to run every day at 2:00), it will be skipped on March 30, 2025, because at 2:00 that day, the time will be moved forward by one hour (due to DST). After the end of DST on October 26, 2025, at 3:00, the time will be shifted back by one hour, but the job in the example will be executed only once.

The network transparency of Ergo Framework allows processes to exchange messages regardless of whether they are running locally or on remote nodes. The process identifier (gen.PID, gen.ProcessID, gen.Alias) includes the name of the node where the process is running—this information is used by the node to route the message.

The Ergo Data Format (EDF) is used for data transmission within Ergo Framework.

EDF - Ergo Framework Data Format

If you're registering a data type that contains another custom type among its child elements, you must register the child type first before registering the parent type.

Example:

All child elements in the registered structures must be public. If you need to encode/decode structures with private fields, you will need to implement a custom encoder/decoder for that data type. To do this, you must implement the edf.Marshaler and edf.Unmarshaler interfaces.

These interfaces allow you to define how the custom type is marshaled (encoded) and unmarshaled (decoded), giving you control over the serialization and deserialization process for types with private fields or special requirements.

Example:

The length of a message encoded using the MarshalEDF method must not exceed 2^32 bytes.

In Golang, the error type is an interface. To transmit error types over the network, they also need to be registered using the edf.RegisterError method. A set of standard errors in Ergo Framework (such as gen.Err*, gen.TerminateReason*) is automatically registered when the node starts.

Additionally, the following limits are imposed on specific types:

• gen.Atom: Maximum length of 255 bytes

• string: Maximum length of 65,535 bytes (2^16)

• binary: Maximum length of 2^32 bytes

• map/array/slice: Maximum number of elements is 2^32

When establishing a connection between nodes, during the handshake phase, the nodes exchange dictionaries containing registered data types and registered errors. These dictionaries are used to reduce the volume of transmitted information during message exchanges, as identifiers are used instead of the names of registered types and errors.

If you register a data type or an error after the connection is established, sending data of that type will result in an error. To make these newly registered types available for network exchange, you will need to disconnect and re-establish the connection.

The EDF format does not support pointers. The use of pointers should remain an internal optimization within your application and should not extend beyond it.

Installation and starting

To install the observer tool, you need to have Golang compiler version 1.20 or higher. Run the following command:

Available arguments for starting observer:

• -help: displays information about the available arguments.

• -version: prints the current version of the Observer tool.

• -host: specifies the interface name for the Web server to run on (default: "localhost").

• -port: defines the port number for the Web server (default: 9911).

• -cookie: sets the default cookie value used for connecting to other nodes.

If you are running observer on a server for continuous operation, it is recommended to use the environment variable COOKIE instead of the -cookie argument. Using sensitive data in command-line arguments is insecure.

After starting observer, it initially has no connections to other nodes, so you will be prompted to specify the node you want to connect to.

Once you establish a connection with a remote node, the Observer application main page will open, displaying information about that node.

If you have integrated the Observer application into your node, upon opening the Observer page, you will immediately land on the main page showing information about the node where the Observer application was launched.

Info (main page)

On this tab, you will find general information about the node and the ability to manage its logging level. Changing the logging level only affects the node itself and any newly started processes, but it does not impact processes that are already running.

Graphs provide real-time information over the last 60 seconds, including the total number of processes, the number of processes in the running state, and memory usage data. Memory usage is divided into used, which indicates how much memory was reserved from the operating system, and allocated, which shows how much of that reserved memory is currently being used by the Golang runtime.

Network (main page)

The Network tab displays information about the node's network stack.

The Mode indicates how the network stack was started (enabled, hidden, or disabled).

The Registrar section shows the properties of the registrar in use, including its capabilities. Embedded Server indicates whether the registrar is running in server mode, while the Server field shows the address and port number of the registrar with which the node is registered.

Additionally, the tab provides information about the default handshake and protocol versions used for outgoing connections.

The Flags section lists the set of flags that define the functionality available to remote nodes.

The Acceptors section lists the node's acceptors, with detailed information available for each. This list will be empty if the network stack is running in hidden mode.

The Connected Nodes section displays a list of active connections with remote nodes. For each connection, you can view detailed information, including the version of the handshake used when the connection was established and the protocol currently in use. The Flags section shows which features are available to the node when interacting with the remote node.

Since the ENP protocol supports a pool of TCP connections within a single network connection, you will find information about the Pool Size (the number of TCP connections). The Pool DSN field will be empty if this is an incoming connection for the node or if the protocol does not support TCP connection pooling.

Graphs provide a summary of the number of received/sent messages and network traffic over the last 60 seconds, offering a quick overview of communication activity and data flow.

Process list (main page)

On the Processes List tab, you can view general information about the processes running on the node. The number of processes displayed is controlled by the Start from and Limit parameters.

By default, the list is sorted by the process identifier. However, you can choose different sorting options:

• Top Running: displays processes that have spent the most time in the running state.

• Top Messaging: sorts processes by the number of sent/received messages in descending order.

• Top Mailbox: helps identify processes with the highest number of messages in their mailbox, which can be an indication that the process is struggling to handle the load efficiently.

For each process, you can view brief information:

The Behavior field shows the type of object that the process represents.

Application field indicates the application to which the process belongs. This property is inherited from the parent, so all processes started within an application and their child processes will share the same value.

Mailbox Messages displays the total number of messages across all queues in the process's mailbox.

Running Time shows the total time the process has spent in the running state, which occurs when the process is actively handling messages from its queue.

By clicking on the process identifier, you will be directed to a page with more detailed information about that specific process.

Log (main page)

All log messages from the node, processes, network stack, or meta-processes are displayed here. When you connect to the Observer via a browser, the Observer's backend sends a request to the inspector to start a log process with specified logging levels (this log process is visible on the main Info tab).

When you change the set of logging levels, the Observer's backend requests the start of a new log process (the old log process will automatically terminate).

To reduce the load on the browser, the number of displayed log messages is limited, but you can adjust this by setting the desired number in the Last field.

The Play/Pause button allows you to stop or resume the log process, which is useful if you want to halt the flow of log messages and focus on examining the already received logs in more detail.

Process information

This page displays detailed information about the process, including its state, uptime, and other key metrics.

The fallback parameters specify which process will receive redirected messages in case the current process's mailbox becomes full. However, if the Mailbox Size is unlimited, these fallback parameters are ignored.

The Message Priority field shows the priority level used for messages sent by this process.

Keep Network Order is a parameter applied only to messages sent over the network. It ensures that all messages sent by this process to a remote process are delivered in the same order as they were sent. This parameter is enabled by default, but it can be disabled in certain cases to improve performance.

The Important Delivery setting indicates whether the important flag is enabled for messages sent to remote nodes. Enabling this option forces the remote node to send an acknowledgment confirming that the message was successfully delivered to the recipient's mailbox.

The Compression parameters allow you to enable message compression for network transmissions and define the compression settings.

Graphs on this page help you assess the load on the process, displaying data over the last 60 seconds.

Additionally, you can find detailed information about any aliases, links, and monitors created by this process, as well as any registered events and started meta-processes.

The list of environment variables is displayed only if the ExposeEnvInfo option was enabled in the node's gen.NodeOptions.Security settings.

Additionally, on this page, you can send a message to the process, send an exit signal, or even forcibly stop the process using the kill command. These options are available in the context menu.

Inspect (process page)

If the behavior of this process implements the HandleInspect method, the response from the process to the inspect request will be displayed here. The Observer sends these requests once per second while you are on this tab.

In the example screenshot above, you can see the inspection of a process based on act.Pool. Upon receiving the inspect request, it returns information about the pool of processes and metrics such as the number of messages processed.

Log (process page)

The Log tab on the process information page displays a list of log messages generated by the specific process.

Please note that since the Observer uses a single stream for logging, any changes to the logging levels will also affect the content of the Log tab on the main page.

Meta-process information

On this page, you'll find detailed information about the meta-process, along with graphs showing data for the last 60 seconds related to incoming/outgoing messages and the number of messages in its mailbox. The meta-process has only two message queues: main and system.

You can also send a message to the meta-process or issue an exit signal. However, it is not possible to forcibly stop the meta-process using the kill command.

Inspect (meta-process page)

If the meta-process's behavior implements the HandleInspect method, the response from the meta-process to the inspect request will be displayed on this tab. The Observer sends this request once per second while you are on the tab.

Log (meta-process page)

On the Log tab of the meta-process, you will see log messages generated by that specific meta-process. Changing the logging levels will also affect the content of the Log tab on the main page.

saturn is a tool designed to simplify the management of clusters of nodes created using the Ergo Framework. It offers the following features:

• A unified registry for node registration within a cluster.

• The ability to manage multiple clusters simultaneously.

• The capability to manage the configuration of the entire cluster without restarting the nodes connected to Saturn (configuration changes are applied on the fly).

• Notifications to all cluster participants about changes in the status of applications running on nodes connected to Saturn.

Installation

To install saturn, use the following command:

Available arguments:

• host: Specifies the hostname to use for incoming connections.

• port: Port number for incoming connections. The default value is 4499.

• path: Path to the configuration file saturn.yaml.

• debug: Enables debug mode for outputting detailed information.

• version: Displays the current version of Saturn.