1 Design Rules for Using the Reactor Effectively #
The ACE_Reactor is a powerful framework for demultiplexing events and dispatching event handlers. Like other frameworks, however, learning to use the ACE_Reactor takes time and effort. One way to shorten the learning curve is to understand the design rules necessary to use the ACE_Reactor effectively. The design rules described below are based on extensive experience gained by helping ACE users program the Reactor framework correctly.
1.1 Understand Concrete Event Handler Return Value Semantics #
Context : The return values of the various handle_* hook methods defined by concrete event handlers cause the ACE_Reactor to behave in different ways. The intent of using return values to trigger different behaviors is to reduce the complexity of the ACE_Reactor's API. However, the return values are often a source of surprise to programmers. Therefore, it is important to understand the effects of the values returned from the handle_* methods, which fall into the following three cases :
- Zero: When a handle_* method returns zero (0) this informs the ACE_Reactor that the event handler wishes to continue being processed as before, i.e., it should remain in a table in the ACE_Reactor's implementation. Thus, the ACE_Reactor will continue to include the handle of this event handler next time it invokes its event demultiplexer via handle events. This is the "normal" behavior of event handlers whose lifetime extends beyond a single handle_* method dispatch.
- Greater than zero: When a handle_* method returns greater than zero (> 0) this informs the ACE_Reactor that the event handler wishes to be dispatched again before the ACE_Reactor blocks on its event demultiplexer. This feature is useful for cooperative event handlers to enhance overall system "fairness". In particular, it allows one event handler to allow other event handlers to be dispatched before it retains control again.
- Less than zero: When a handle_* method returns less than zero (< 0) this informs the ACE_Reactor that the event handler wants to be closed and removed from the ACE_Reactor's internal tables. To accomplish this, the ACE_Reactor invokes the event handler's handle_close cleanup method. This method can perform user-defined termination activities, such as deleting dynamic memory allocated by the object or closing log files. When the handle close method returns, the ACE_Reactor removes the associated concrete event handler from its internal tables.
Design rule 0: Do not manually delete event handler objects or call handle close explicitly - Instead, ensure method automatically. Thus, applications must follow the proper protocol, i.e., either by (1) returning a negative value from a handle_* hook method or (2) calling remove handler. This design rule ensures that an ACE_Reactor can cleanup its internal tables properly. If this rule is not obeyed, the ACE_Reactor will incur unpredictable memory management problems when it later tries to remove externally deleted concrete event handlers. Subsequent design rules
elaborate on how to ensure that the ACE_Reactor invokes the handle_close cleanup method.
Design rule 1: Return expressions in handle_* methods of classes inheriting from ACE_Event_Handler should be constant. This design rule makes it easier to statically check whether the handle_* methods are returning appropriate values. If this rulemust be violated, developers must precede the return statement with a comment that explains why a variable is used rather than a constant.
Design rule 2: Return statements in handle_* methods of classes inheriting from ACE_Event_Handler that do not return 0 must be preceded by a comment stating what the return value signifies. This design rule ensures that all non-0 return values are explicitly intended by developers.
1.2 Understand the handle close() Cleanup Hook Semantics #
Context: The handle close cleanup hook method must be called by the ACE_Reactor either (1) implicitly, i.e., when a handle_* method returns a negative value like 1 or (2) explicitly, i.e., if an application calls the remove handler method to remove a concrete event handler. In particular, the ACE_Reactor will not call handle close automatically when an I/O handle is closed, either by the local application or a remote application. Thus, applications must determine when an I/O handle has closed down and must take the appropriate steps so the Reactor will trigger the handle close cleanup ACE method.
Example: The following Logging Handler code fragment from Section 4.2.2 illustrates how to trigger the cleanup hook incorrectly:
// Hook method for handling the reception of
// remote logging transmissions from clients.
int Logging_Handler::handle_input (ACE_HANDLE)
{
return peer_stream_.recv (&len, sizeof len);
}
Note that this method will only trigger the handle close hook when recv fails, i.e., returns 1. However, it will not work correctly when recv returns 0 or > 0. To minimize problems with handle close cleanup methods, therefore, observe the following design rules when implementing concrete
event handlers:
Design rule 3: Return a negative value from a handle_* method when you want to trigger the corresponding handle close cleanup method on the concrete event handler. The following revised Logging Handler code fragment illustrates how to trigger the cleanup hook correctly:
// Hook method for handling the reception of
// remote logging transmissions from clients.
int Logging_Handler::handle_input (ACE_HANDLE)
{
ssize_t n = peer_stream_.recv (&len, sizeof len);
if (n == 0)
// Trigger handle_close().
return -1;
// ...
// Keep handler registered for "normal" case.
return 0;
}
When the handle input method receives a 0 from recv, it returns 1. This value triggers the ACE_Reactor to call the handle close cleanup hook.
The value 1 is typically used to trigger the cleanup hook since it's a common error code with the ACE_OS wrappers for native OS APIs. However, any negative value from a handle_* method will trigger handle close.
Design rule 4: Confine all Event Handler cleanup activities to the handle close cleanup method. In general, it is easier to consolidate all the cleanup activities in the handle close method, rather than dispersing them throughout the handle_* methods or other methods in an event handler. This design rule is particularly important to follow when dealing with dynamically allocated event handlers that must be cleaned up with delete this (see Rule 9).
1.3 Remember that ACE Time Value Arguments are Relative #
Context: It's important to remember that both ACE Time Value arguments passed to the schedule timer method of the ACE_Reactor are specified relative to the current time.
Example: The following code schedules an object to print the name of the executable program, i.e., argv[0], every interval number of seconds, starting delay seconds in the future:
class Hello_World : public ACE_Event_Handler
{
public:
virtual int handle_timeout (const ACE_Time_Value &tv, const void *act)
{
ACE_DEBUG ((LM_DEBUG, "%[s] %d, %d\n", act, tv.sec (), tv.usec ()));
return 0;
}
};
int main (int argc, char *argv[])
{
if (argc != 3)
ACE_ERROR_RETURN ((LM_ERROR, "usage: %s delay interval\n", argv[0]), -1);
Hello_World handler; // timer object.
ACE_Time_Value delay = ACE_OS::atoi (argv[1]);
ACE_Time_Value interval = ACE_OS::atoi (argv[2]);
// Schedule the timer.
ACE_Reactor::instance ()->schedule_timer(&handler, (const void *) argv[0], delay, interval);
// Run the event loop.
for (;;)
ACE_Reactor::instance ()->handle_events ();
/* NOTREACHED */
}
A common mistake is to pass an absolute time value to schedule timer. For instance, consider a different example:
ACE_Time_Value delay = ACE_OS::atoi (argv[1]); delay += ACE_OS::gettimeofday (); // Callback every following 10 seconds. ACE_Time_Value interval = delay + 10; ACE_Reactor::instance ()->schedule_timer (&handler, 0, delay, interval);
However, this timer will not expire for a long time in the future since it adds the current time of day to the delay and interval requests by the user. The following is a design rule to follow when implementing concrete event handlers to minimize problems with absolute ACE Time Values:
Design rule 5: Do not use absolute times as the third or fourth arguments to ACE_Reactor::schedule timer. In general, these arguments should be less than an extremely long delay, which is significantly less than the current time.
1.4 Track ACE Event Handler Lifetimes Carefully #
Various problems arise from failing to track the lifetimes Handlers that are registered with an of ACE Event ACE_Reactor. These problems are hard to track down without the use of a memory error detection tool like Purify 23, which catches some, but not all, of the following lifetime-related problems:
1.4.1 Use Non-dynamically Allocated Event Handlers Sparingly #
Context: Certain types of applications, such as embedded real-time systems, try to minimize the use of dynamic memory, e.g., to make system performance more predictable. It's important to be very careful to use nondynamically allocated concrete event handlers correctly with an ACE_Reactor.
Example: Consider the following concrete event handler definition:
class My_Event_Handler : public ACE_Event_Handler
{
public:
ACE_Reactor_Mask = ACE_Event_Handler::READ_MASK)
My_Event_Handler (const char *str = "hello") : str_ (ACE_OS::strnew (str)) {}
virtual int handle_close (ACE_HANDLE = ACE_INVALID_HANDLE,
{
// Commit suicide.
delete this;
}
~My_Event_Handler (void) {
delete [] this->str_;
}
// ...
private:
char *str_;
};
This class deletes itself when it's removed from the ACE_Reactor via its handle close cleanup method. Although this may look somewhat unconventional, it is a perfectly valid C++ idiom. However, it only works as long as the object being deleted was allocated dynamically.
In contrast, if the object being deleted was not allocated dynamically, the global dynamic memory heap will be corrupted.
The reason is that the delete operator will interpret this as a valid address in the heap. This will cause subtle memory management problems when the delete operator tries to insert the non-heap memory into its internal freelist. The following example illustrates a common use-case that
can corrupt the heap:
int main (void)
{
// Non-dynamically allocated.
My_Event_Handler my_event_handler;
ACE_Reactor::instance ()->register_handler (&my_event_handler, ACE_Event_Handler::READ_MASK);
// ...
// Run event-loop.
while (/* ...event loop not finished... */)
ACE_Reactor::instance ()->handle_events ();
// The <handle_close> method deletes an
// object that wasn't allocated dynamically...
ACE_Reactor::instance ()->remove_handler (&my_event_handler, ACE_Event_Handler::READ_MASK);
return 0;
}
The problem with the code above is that the Reactor will invoke the handle close method of Handler when remove handler is called. Unfortunately, the handle close method will delete this on the my event handler object, which was not allocated dynamically. One way to guard against this problem is to place the destructor in the private section of the My Event Handler class, i.e.:
class My_Event_Handler : public ACE_Event_Handler
{
public:
My_Event_Handler (const char *str);
// ...
private:
// Place destructor into the private section
// to ensure dynamic allocation.
~My_Event_Handler (void);
// ...
};
In this class, the My Event Handler destructor is placed in the private access control section of the class. This C++ idiom ensures that all instances of this class must be allocated dynamically. If an instance is accidentally defined as a static or auto, it will be flagged as an error at compiletime.
The following is a design rule to follow when implementing concrete event handlers to minimize problems with concrete event handler lifetimes:
Design rule 6: Do not delete event handlers that were not allocated dynamically. Any handle close method that contains delete this and whose class does not have a private destructor may be in violation of this design rule. In the absence of a convention checker that can identify this case statically, the delete this should be preceded immediately by a comment explaining why this idiom is used.
1.4.2 Remove Concrete Event Handler Appropriately #
Context: As described above, determining how to remove concrete event handlers from an ACE_Reactor can be tricky. The following examples illustrate some other common problems.
Example: The following program illustrates another common problem related to the lifetimes of concrete event handlers:
ACE_Reactor reactor;
int main (void)
{
My_Event_Handler my_event_handler;
ACE_Reactor::instance ()->register_handler (&my_event_handler, ACE_Event_Handler::READ_MASK);
while (/* ...event loop not finished... */)
ACE_Reactor::instance ()->handle_events ();
// The destructor of the ACE_Reactor singleton
// will be called when the process exits. It
// removes all registered event handlers.
return 0;
}
The lifetime of my_event_handler is defined by the lifetime of the main function. In contrast, the lifetime of the ACE_Reactor singleton is defined by the lifetime of the process. Thus, when the process exits, the destructor for the reactor will be called. (This is ensured by the ACE Object Manager, which destroys all singletons in ACE before a process exits.) the ACE_Reactor removes all event handlers that are still registered by calling their handle_close method. If my event handler is still registered with the reactor, however, its handle_close method will be called after the object has gone out of scope and been destroyed. The following are three more design rules to follow when implementing concrete event handlers to minimize problems with concrete event handler lifetimes:
Design rule 7: Always allocate concrete event handlers dynamically from the heap. This is a relatively straightforward solution to many of problems related to the lifetime of concrete event handlers. If it is not possible to follow this rule, a comment must be provided when the concrete event handler is registered with the ACE_Reactor explaining why dynamic allocation is not used. This comment should appear immediately before the register handler statement that registers the statically allocated concrete event handler with the ACE_Reactor.
Design rule 8: Remove ACE Event Handlers from their associated ACE_Reactor before exiting the scope where they are "live". This rule should be used in cases where Rule 7 is not followed.
Design rule 9: Allow the delete this idiom in the handle close method only, i.e., do not allow delete this in the other handle_* methods or in other methods in the event handler. This rule makes it easier to check whether there are potential problems with deleting non-dynamically allocated memory. It also ensures that the ACE_Reactor doesn't try to access a pointer to a deleted event handler. Naturally, components unrelated to the ACE_Reactor may have different rules governing selfdeletion.
Design rule 10: Only delete this when the final registered event has been removed from an ACE_Reactor for a concrete event handler. This rule avoids "Dangling pointers" that can otherwise occur by prematurely deleting a concrete event handler that registered with an ACE_Reactor for multiple events.
For instance, the my event handler could be registered both for READ and WRITE events, as follows:
ACE_Reactor::instance ()->register_handler (&my_event_handler, ACE_Event_Handler::READ_MASK | ACE_Event_Handler::WRITE_MASK);
In this case, when the handle input method returns -1 the ACE_Reactor will invoke the handle_close cleanup hook method. This method must not delete this until the WRITE_MASK is also removed for that concrete event handler, e.g., by having it return a negative value or explicitly removing it via
ACE_Reactor::instance ()->remove_handler (&my_event_handler, ACE_Event_Handler::WRITE_MASK);
The following method illustrates one way to keep track of this information:
class My_Event_Handler : public ACE_Event_Handler
{
public:
My_Event_Handler (void)
{
}
virtual int handle_close (ACE_HANDLE h,
{
}
// ... handle_input() and handle_output() methods.
private:
ACE_Reactor_Mask mask_;
// Keep track of when to delete this.
The solution above maintains an ACE_Reactor Mask that keeps track of when all events a concrete event handler is registered for have been removed from an ACE_Reactor.
== Beware of WRITE MASK Semantics ==
Context: The WRITE MASK can be used to instruct the ACE ever an application can write to an I/O handle without blocking. The following code illustrates how to register an
event handler using the WRITE MASK:
{{{
ACE_Reactor::instance ()->mask_ops(event_handler,
It is always ok to write to a handle unless the connection is flow controlled. Thus, this ACE_Reactor will Reactor to callback to an event handler when-
// Keep track of which bits are enabled.
ACE_SET_BITS (this->mask_, ACE_Event_Handler::READ_MASK | ACE_Event_Handler::WRITE_MASK);
// Register ourselves with the Reactor for both READ and WRITE events.
ACE_Reactor::instance ()->register_handler(this, this->mask_);
ACE_Reactor_Mask mask)
if (mask == ACE_Event_Handler::READ_MASK) {
ACE_CLR_BITS (this->mask_,
ACE_Event_Handler::READ_MASK);
// Perform READ_MASK cleanup logic.
}
else if (mask == ACE_Event_Handler::WRITE_MASK) {
ACE_CLR_BITS (this->mask_,
ACE_Event_Handler::WRITE_MASK);
// Perform WRITE_MASK cleanup logic.
}
// Only delete ourselves if we뭭e been closed
// down for both READ and WRITE events.
if (this->mask_ == 0)
delete this;
Reactor.
ACE_Event_Handler::WRITE_MASK,
ACE_Reactor::ADD_MASK);
keep calling back the handle output method of the
event handler until (1) the connection flow controls
or (2) the mask ops method is instructed to clear the
WRITE MASK.
Example: A common programming mistake is to forget
to clear the WRITE MASK once there뭩 no longer any
more data to write to the connection. This omission
will cause the ACE_Reactor to continuously invoke the
handle output method of the concrete event handler
that뭩 registered with the WRITE MASK. Therefore, the following
design rule should be followed to avoid this problem:
Design rule 11: Clear the WRITE MASK when you no
longer want the concrete event handler뭩 handle output
method to get called back.
The following code illustrates how to ensure the
handle output method is no longer called back:
ACE_Reactor::instance ()->mask_ops
(event_handler,
ACE_Event_Handler::WRITE_MASK,
ACE_Reactor::CLR_MASK);
The ACE_Reactor defines a short-hand method for accomplishing
the same thing:
ACE_Reactor::instance ()->cancel_wakeup
(event_handler,
ACE_Event_Handler::WRITE_MASK);
These methods are typically called when there are no more
output messages pending on a concrete event handler.
To facilitate automated checking of this rule, programmers
must insert comments into their handle output methods.
These comments will indicate which return paths are not
intended to clear the WRITE MASK, i.e., the event handler
wants to continue to be called back when it뭩 뱋k to write.?
Likewise, programmers should also comment the path(s)
where the WRITE MASK is removed. If there are no paths in
the handle output method that clear the WRITE MASK,
this indicates a potential violation of this design rules.
For example, the following handle output method illustrates
a potential application of this design rule:
int
My_Event_Handler::handle_output (ACE_HANDLE)
{
if (/* output queue is now empty */) {
/* Removing WRITE_MASK */
ACE_Reactor::instance ()->cancel_wakeup
(event_handler,
ACE_Event_Handler::WRITE_MASK);
return 0;
} else {
// ... continue to transmit messages
// from the output queue.
/* Not removing WRITE_MASK */
return 0;
}
}
If there were no comments indicating the WRITE MASK is
removed this would be a violation of the design rule.
17
5.6 Register Concrete Event Handlers Appropriately
Context: There are two ways to register a concrete event
handler with an ACE_Reactor for I/O operations:
Explicitly pass the handle: This approach uses the
following ACE_Reactor method:
int register_handler
(ACE_HANDLE io_handle,
ACE_Event_Handler *event_handler,
ACE_Reactor_Mask mask);
and passes the ACE HANDLE of the I/O device explicitly,
i.e.:
void register_socket (ACE_HANDLE socket,
ACE_Event_Handler *handler)
{
}
ACE_Reactor::instance ()->register_handler
(socket,
handle: This
5.7 Remove Closed Handles/Handlers from
the Reactor by Centralizing Cleanup
ACE
ACE
Context: When a connection is closed down, the handle
is no longer valid for I/O. In this case, select will continue
to report that the handle is 뱑eady for reading?so that
the handle can be closed, which is typically done by calling
crete event handler뭩 handle close cleanup method.
Example: A common mistake when writing applications
with the Reactor is to fail to remove defunct handles and
their associated event handlers from the ACE_Reactor. If
you fail to do this, however, the ACE_Reactor will continually
callback the handle input method on the event
handler until the handle and its handler are removed from the
ACE_Reactor. The following design rule helps to avoid
this problem:
Design rule 13: Return a negative value from the
handle_* methods when a connection closes down (or
when an error occurs on the connection) and centralize
cleanup activities in the handle close method.
Code that follows this design rule is usually structured as
follows:
int handle_input (ACE_HANDLE handle)
{
// ...
ssize_t result =
if (result <= 0)
else
// ...
When the 1 is returned, the ACE_Reactor will call
your handle close cleanup method. To avoid resource
leaks, make sure this method gives the event handler
a chance to delete itself and close its handle, e.g.,
turns, the ACE_Reactor will automatically remove the
handle/handler tuple from its internal table if the handle
is no longer registered for any types of events.
Tasks
OS::close or ACE OS::closesocket in a con-
ACE_OS::read (handle, buf, bufsize);
// Connection has closed down or an
// error has occurred.
return -1;
OS::close (handle). Once handle close re-
Reactor, the default get handle in the
handler,
ACE_Event_Handler::READ_MASK);
the pass
handle method, which returns 1
Handler base class. If you don뭪 follow
Handler base class will return 1, which is ACE Event
if they call remove handler within the handle close
// ...
Note that this register handler method allows the
same concrete event handler to be registered with multiple
ACE HANDLEs. This feature of the Reactor framework
makes it possible to minimize the amount of state required
to handle many clients that are connected simultaneously to
the same event handler.
Implicitly
approach uses the other register handler method on
the ACE_Reactor:
int register_handler
(ACE_Event_Handler *event_handler,
ACE_Reactor_Mask mask);
In this case, a 밺ouble-dispatch?[6] is performed by the
Reactor to obtain the underlying ACE HANDLE from ACE
the concrete event handler via its get handle method.
This method is defined with the following signature in the
ACE Event Handler base class:
virtual ACE_HANDLE get_handle (void) const;
Example: When using implicit registration, a common
mistake is to omit the const on the get handle method
when deriving from ACE Event Handler. Omitting the
const prevents the compiler from properly overriding the
get handle method in the subclass. Instead, it will hide
the method in the subclass, thereby generating code that will
call the base class get
by default. Therefore, it is important to obey the following
design rule:
Design rule 12: Make sure the signature of the
handle method is consistent with the one in the get
ACE Event
this rule, and you 밿mplicitly?pass the ACE HANDLE
to the ACE
erroneous.
5.8 Use the DONT CALL Flag to Avoid Recursive
handle close() Callbacks
Context: Earlier rules covered how an event handler뭩
handle close hook is automatically invoked by an
ACE_Reactor when the handler is removed either explicitly,
e.g., when the remove handler method is called,
or implicitly, e.g., by returning a negative value from a
handle_* method. Applications must be careful, however,
cleanup method since this can trigger infinite recursion.
18
Example: The following handle close method will
infinitely recurse since the remove handler method will
reinvoke handle close again:
int
My_Event_Handler::handle_close
(ACE_HANDLE,
ACE_Reactor_Mask)
{
// ...
ACE_Reactor::instance ()->remove_handler
(this->get_handle (),
// Remove all the events for which we뭨e
// registered.
ACE_Event_Handler::RWE_MASK);
// ...
}
The following design rule prevents infinite recursion from
occurring.
Design rule 14: Always pass the DONT CALL flag to
remove handler when calling it in a handle close
method. This rule ensures that the ACE_Reactor will not
invoke the handle close method recursively. The following
code illustrates how to do this:
int
My_Event_Handler::handle_close
(ACE_HANDLE,
ACE_Reactor_Mask)
{
// ...
ACE_Reactor::instance ()->remove_handler
(this->get_handle (),
// Remove all the events for which we뭨e
// registered. We must pass the DONT_CALL
// flag here to avoid infinite recursion.
ACE_Event_Handler::RWE_MASK |
ACE_Event_Handler::DONT_CALL);
// ...
}
Incidentally, remove handler is typically called from
within handle close in situations where (1) a concrete
event handler is registered for multiple events and (2) the
first time handle close is called should trigger a complete
shutdown of the event handler. Thus, it뭩 essential
that handle close also remove the other events this event
handler is registered for with the ACE_Reactor.
5.9 Don뭪 Overload the ACE Event Handler
handle_*() methods
Context: It is generally not a good idea for subclasses to
overload virtual methods inherited from a base class. In C++,
this overloading 밾ides?the inherited function in subclass,
i.e.,
class My_Event_Handler :
public ACE_Event_Handler
{
// Overload the base classes?method:
// virtual int handle_input (ACE_HANDLE)
virtual int handle_input (void);
}
In this example, the My Event Handler class overloads
the handle input method, which is defined in the
ACE Event Handler base class. However, this function
has a side-effect of 밾iding?the method in the base class,
which is usually undesirable.
Example: Although overloading of base class methods
is problematic with C++ applications in general,
it is particularly tricky for classes that subclass from
Event Handler. In particular, application develop- ACE
ers may not realize that the handle input method on an
My Event Handler object will not be dispatched when
input events occur on the ACE_Reactor that this object is
registered with because the signatures do not match exactly.
To avoid this problem altogether, simply abide by the following
design rule:
Design rule 15: Do not overload any handle_* methods
in ACE Event Handler subclasses. Naturally, it is fine to
override these methods as necessary in order to customize
the behavior of the desired handle_* hooks. Many C++
compilers warn about this behavior, but it뭩 generally good
to avoid it whenever possible to avoid confusion.
5.10 Beware of Deadlock in Multi-threaded
Reactor Applications
Context: Although Reactor뭩 are generally used to implement
single-threaded concurrent applications, they also can
be used to in multi-threaded applications. In this context, it뭩
important to beware of deadlock between multiple threads
that are sharing a common ACE_Reactor.
Example: When the ACE_Reactor dispatches a callback
to an event handler뭩 handle_* method it holds
an ACE Token for the duration of the callback. This
ACE Token is defined as a recursive mutex [24], which
keeps track of the identify of the thread that holds the mutex
in order to avoid 뱒elf-deadlock.?If the dispatched
handle_* method directly or indirectly calls back into
the ACE_Reactor within the same thread of control, the
ACE Token뭩 acquire method detects this automatically
and simply increases its count of the lock recursion nesting
depth, rather than deadlocking the thread.
Even with recursive mutexes, however, it is still possible
to incur deadlock if (1) the original handle_* callback
method makes a blocking call to a method in another thread
and (2) that method in the second thread directly or indirectly
calls into the same ACE_Reactor. In this case, deadlock
occurs since the ACE_Reactor뭩 ACE Token does not realize
that the second thread is calling on behalf of the first
thread where the handle_* hook method was originally
dispatched.
To avoid deadlock when using the ACE_Reactor in a
multi-threaded application, try to apply the following design
rule:
Design rule 16: Do not make blocking calls to other
threads in handle_* methods if these threads will directly
19
or indirectly call back into the same ACE_Reactor. It
may be necessary to use an ACE Message Queue to exchange
information asynchronously if a handle_* callback
method must communicate with another thread that accesses
the same ACE_Reactor.
5.11 Ensure Controlling Thread Owns Eventloop
in Multi-threaded Reactor Applica-
}
}
};
tions
Context: Only one thread of control at a time can invoke
the handle events method on an ACE_Reactor. To
ensure this, each ACE_Reactor keeps track of the thread
of control that 뱋wns?its event loop. By default, the owner
of an ACE_Reactor is the identifier of the thread that initialized
it. There are certain use-cases, however, where
the thread that initializes an ACE_Reactor is different
from the thread that ultimately runs its event-loop via the
handle events method.
Example: Consider the following erroneous C++ code
fragment:
class My_Server :
public ACE_Task<ACE_NULL_SYNCH>
{
public:
// Initializer.
int open (size_t size) {
// This call sets the owner
// of the event-loop to
// <ACE_OS::thr_self>.
reactor_.open (size);
// Make this an Active Object
// (inherited from <ACE_Task>).
this->activate ();
// ...
// Hook method entry point for
// the Active Object. This runs
// in a new thread of control.
int svc (void) {
// ...
// This call will return -1
// since the new thread of
// control doesn뭪 몣own뮃 this
// <reactor_> instance.
reactor_.handle_events ();
private:
ACE_Reactor reactor_;
Since My Server inherits from ACE Task, calling its
activate method will make the My Server object become
an Active Object [25], which runs its svc hook
method entry point in a separate thread of control. Therefore,
the handle events method will fail since it still thinks
the owner of this ACE_Reactor is the original thread that
initialized it.
It is straightforward to fix this code by simply having
the new thread of control become the owner of the
Reactor, as follows: ACE
20
int svc (void) {
// ...
// Have this thread become the
// new owner of <reactor_>뭩 event-loop.
reactor_.owner (ACE_OS::thr_self ());
// Now this call works correctly.
reactor_.handle_events ();
// ...
}
Design rule 17: Ensure that the thread running an
ACE_Reactor뭩 event loop becomes the controlling thread
by assigning its thread identifier via the owner method.
The one exception to this rule is if the thread running the
handle events method is the same thread that created
this particular instance of the ACE_Reactor.
6 Concluding Remarks
The ACE_Reactor is an OO framework designed to simplify
the development of concurrent, event-driven distributed
applications. By encapsulating low-level OS event demultiplexing
mechanisms within an OO C++ interface, the
ACE_Reactor makes it easier to develop correct, compact,
portable, and efficient applications. Likewise, by separating
policies and mechanisms, the ACE_Reactor enhances
reuse, improves portability, and provides transparent extensibility.
The following C++ language features are used to enhance
to the design of the ACE_Reactor and its applications.
Classes: The encapsulation provided by C++ classes improves
portability. For instance, the ACE_Reactor class
shields applications from differences between OS event
demultiplexers like WaitForMultipleObjects and
select.
Objects: Registering concrete event handler objects, rather
than stand-alone functions, with the ACE_Reactor helps
integrate application-specific state together with the methods
that use this state.
Inheritance and dynamic binding: These features facilitate
transparent extensibility by allowing developers to enhance
the functionality of the ACE_Reactor and its associated
applications without modifying existing code.
Templates: C++ parameterized types help increase the
reusability by factoring variability into uniform classes,
which can be 뱎lugged?into the generic templates.
For instance, the ACE Acceptor can be instantiated
with SVC HANDLERs other than Logging Handler and
PEER ACCEPTORs other than ACE SOCK Acceptor.
Perhaps the greatest challenge to using the
ACE_Reactor is that its 밿nversion of control?programming
model makes it hard to determine where an application뭩
main flow of control is executing. This is a common
challenge with other callback-based dispatcher mechanisms,
such as the X-windows event loop.
The C++ source code and documentation for the
Reactor and ACE socket wrappers is available at
Acknowledgements
Thanks to Hans Rohnert and Bill Landi of Siemens for their
helpful comments on this paper.
ACE
www.cs.wustl.edu/schmidt/ACE.html. Also
included with this release are a suite of test programs and
examples, as well as many other C++ wrappers that encapsulate
named pipes, STREAM pipes, mmap, and the System
V IPC mechanisms (i.e., message queues, shared memory,
and semaphores).
th
2 References #
[1] D. C. Schmidt, “Reactor: An Object Behavioral Pattern for Concurrent Event Demultiplexing and Event Handler Dispatching,” in Pattern Languages of Program Design (J. O. Coplien and D. C. Schmidt, eds.), pp. 529–545, Reading, Massachusetts: Addison-Wesley, 1995.
[2] D. C. Schmidt, “ACE: an Object-Oriented Framework for Developing Distributed Applications,” in Proceedings of the USENIX C++ Technical Conference, (Cambridge, Massachusetts), USENIX Association, April 1994.
[3] W. R. Stevens, UNIX Network Programming, Volume 1: Networking APIs: Sockets and XTI, Second Edition. Englewood Cliffs, NJ: Prentice Hall, 1998.
[4] H. Custer, Inside Windows NT. Redmond, Washington: Microsoft Press, 1993.
[5] D. C. Schmidt and C. Cleeland, “Applying Patterns to Develop Extensible ORB Middleware,” IEEE Communications Magazine, vol. 37, April 1999.
[6] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software. Reading, Massachusetts: Addison-Wesley, 1995.
[7] D. C. Schmidt and P. Stephenson, “Experiences Using Design Patterns to Evolve System Software Across Diverse OS Platforms,” in Proceedings of the 9th European Conference on Object-Oriented Programming, (Aarhus, Denmark), ACM, August 1995.
[8] D. C. Schmidt, “Transparently Parameterizing Synchronization Mechanisms into a Concurrent Distributed Application,” C++ Report, vol. 6, July/August 1994.
[9] I. Pyarali, T. H. Harrison, and D. C. Schmidt, “Asynchronous Completion Token: an Object Behavioral Pattern for Efficient Asynchronous Event Handling,” in Pattern Languages of Program Design (R. Martin, F. Buschmann, and D. Riehle, eds.), Reading, Massachusetts: Addison-Wesley, 1997.
10 R. E. Barkley and T. P. Lee, “A Heap-Based Callout Implementation to Meet Real-Time Needs,” in Proceedings of the USENIX Summer Conference, pp. 213–222, USENIX Association, June 1988.
11 D. C. Schmidt, D. L. Levine, and S. Mungee, “The Design and Performance of Real-Time Object Request Brokers,” Computer Communications, vol. 21, pp. 294–324, Apr. 1998.
12 D. E. Comer and D. L. Stevens, Internetworking with TCP/IP Vol II: Design, Implementation, and Internals. Englewood Cliffs, NJ: Prentice Hall, 1991.
13 G. Varghese and T. Lauck, “Hashed and Hierarchical Timing Wheels: Data Structures for the Efficient Implementation of a Timer Facility,” in The Proceedings of the Symposium on Operating System Principles, November 1987.
14 J. Hu, S. Mungee, and D. C. Schmidt, “Principles for Developing and Measuring High-performance Web Servers over ATM,” in Proceedings of INFOCOM ’98, March/April 1998.
15 J. Hu, I. Pyarali, and D. C. Schmidt, “Measuring the Impact of Event Dispatching and ConcurrencyModels onWeb Server Performance Over High-speed Networks,” in Proceedings of the 2nd Global Internet Conference, IEEE, November 1997.
16 D. C. Schmidt, “IPC SAP: An Object-Oriented Interface to Interprocess Communication Services,” C++ Report, vol. 4, November/December 1992.
17 D. L. Presotto and D. M. Ritchie, “Interprocess Communication in the Ninth Edition UNIX System,” UNIX Research System Papers, Tenth Edition, vol. 2, no. 8, pp. 523–530, 1990.
18 G. Booch, Object Oriented Analysis and Design with Applications (Edition). Redwood City, California: Benjamin/Cummings, 1994.
19 D. C. Schmidt, “Acceptor and Connector: Design Patterns for Initializing Communication Services,” in Pattern Languages of Program Design (R. Martin, F. Buschmann, and D. Riehle, eds.), Reading, Massachusetts: Addison-Wesley, 1997.
20 D. C. Schmidt and T. Suda, “Transport System Architecture Services for High-Performance Communications Systems,” IEEE Journal on Selected Areas in Communication, vol. 11, pp. 489–506, May 1993.
21 A. Koenig, “When Not to Use Virtual Functions,” C++ Journal, vol. 2, no. 2, 1992.
22 T. H. Harrison, D. L. Levine, and D. C. Schmidt, “The Design and Performance of a Real-time CORBA Event Service,” in Proceedings of OOPSLA ’97, (Atlanta, GA), pp. 184–199, ACM, October 1997.
23 P. S. Inc., Purify User’s Guide. PureAtria Software Inc., 1996.
24 D. C. Schmidt, “An OO Encapsulation of Lightweight OS Concurrency Mechanisms in the ACE Toolkit,” Tech. Rep. WUCS-95-31, Washington University, St. Louis, September 1995.
25 R. G. Lavender and D. C. Schmidt, “Active Object: an Object Behavioral Pattern for Concurrent Programming,” in Pattern Languages of Program Design (J. O. Coplien, J. Vlissides, and N. Kerth, eds.), Reading, Massachusetts: Addison-Wesley, 1996.








![[http]](/wiki/imgs/http.png)
