Created attachment 90027 [details]
Zip of the example API as a module project with some included actions

To see the example as I'd like for people to view the behavior of blocking the EDT, you just need to run the project.
After the RCP appears, go to the View menu and select "Show Example (non blocking)" to see a dialog open with
a progress bar counting in it.  Move this window to the side of your screen. 

Go right back to the View menu and also select "Show Example (blocking)" to see another example.  Notice the delay in
opening, and the stopped progress on the other dialog (The EDT is blocked and is not processing the other events).  How
much is the EQ growing while this blocking happens?  The EDT-global lock is probably asserted, just stopping everything.

When the timeout period expires in the runOffAWTImpl class, two dialogs will appear stacked on top of each other (should
there be an issue opened for that?).  Unstack the dialogs and you can see the other progress bar.  Pushing a button on
either of these two dialogs will close both, cancelling that action.

The other action can be cancelled too by pushing its dialog's button.

Just for reference, the runOffAWT work is issue 170882.

Created attachment 90141 [details]
Updated with some API changes and Java Doc updates

I've included some tests, a word document with some of my views of why this API should exist.  There is an additional
example action that I am filling out with some more complex example usage.

I'm looking at this code, but I confess I'm having a hard time deciphering what it's all for. It's very conceptually dense, and not obvious at all what all of these Control* classes do.

I would suggest *not* looking to SwingWorker for inspiration - it is not cleanly designed.  For some comments on why, see http://weblogs.java.net/blog/2008/08/11/capability-pattern-future-proof-your-apis - I see similar patterns here (for example, why should ControlProcessor.call() be world-invokable).

*If* I understand all the things this code is supposed to do (and that's a big if), I think you could make this all much simpler for the user without losing functionality.  For example, publishing interim results is really orthangonal to running a background task - pass something as an argument to your runnable-equivalent which has asynchronous and synchronous publish methods that take a runnable.

If the goal is to manage a series of background tasks, which occasionally publish interim work products on the EDT (and incidentally also support cancellation), you could probably do this with an API of 2-3 API classes, a small SPI, and even do it outside the Progress API altogether.

Instead of annotations for on/off EDT, consider simply passing in an object to your run method which you can pass a Runnable to publish something.  Each step of background work can be transactional for cancellation semantics; they can be implemented as a linked list, with each step being one progress step so they are composable. Very, very rough sketch w/o failure handling, etc.:

public abstract class Transaction<ArgType, ResultType> {
   protected Transaction(String name, Class<ArgType> resultType, Class<ResultType> argType);
   public abstract void run (TransactionController controller, ArgType arg);
   public abstract void rollback(TransactionController controller, ArgType arg);
   public <R> Transaction<T,R> add (Transaction<R> next);
}
public final class TransactionController { //probably backed by injected impl
   TransactionController (FailureHandler handler) {
   }
   public void publish (Runnable r);
   public void publishSynchronously (Runnable r);
   public void failed (Transaction<?> xaction, Throwable failure);
   TransactionUI ui();
}
public interface FailureHandler {
    void failed(Transaction xaction, Throwable e, String msg);
}
public abstract class TransactionUI {
    static TransactionUI createUI(boolean blocking) { // look up a factory for TransactionUIs which delegates to progress API
    }
    public abstract void onBeginTransaction (Transaction transaction);
    public abstract void onEndTransaction(Transaction transaction);
    public abstract void onBeginRollback(Transaction transaction);
    public abstract void onFinishRollback(Transaction transaction);
    public abstract void onFailure (Transaction xaction, boolean isRollback);
}

You can then chain up a series of tasks:
Transaction<A,C> xaction = new MyTransactionThatProducesB().add(new MyTransactionThatProducesC());
xaction.start (new A(), new FailureHandler() {...});

An SPI can implement a factory over TransactionUIs which delegates to the Progress API.

As far as I can decipher it, this would accomplish about the same thing, but be conceptually considerably simpler.

A few nitpicks from reading over the code:

TDB01: Call ControllableAction something other than *Action.  Yes, we have FileSystem.AtomicAction and Mutex.Action, but these names are confusing to API users who expect them to have something to do with javax.swing.Action

TDB02: Sources are uncompilable - H:\work\EDTControl\src\org\wonderly\edt\examples\ListListModel.java:358: name clash: <T#1>toArray(T#1[]) and toArray(T#2[]) have the same erasure
        public <T> T[] toArray(T[] a) {
  where T#1,T#2 are type-variables:
    T#1 extends Object declared in method <T#1>toArray(T#1[])
    T#2 extends Object declared in class ListListModel
(can be fixed by using non-generified overrides)

TDB03: In ExampleActionNoBlocking, prefer RequestProcessor.getDefault().post(ca) to new Thread( ca ).start();

TDB04: Prefer EventQueue.isDispatchThread() to SwingUtilities version - the former is (or was as of JDK 5) faster

TDB05: What is the javax.security.auth.Subject stuff in DefaultControlContext and what is it for?

TDB06: View > Show Example throws: java.lang.IllegalStateException: Can not use EDT to wait for results
Caused: java.util.concurrent.ExecutionException
	at org.netbeans.modules.edt.ControllableAction.get(ControllableAction.java:402)
	at org.wonderly.edt.examples.ExampleActionNoBlocking.actionPerformed(ExampleActionNoBlocking.java:77)
	at org.openide.awt.AlwaysEnabledAction$1.run(AlwaysEnabledAction.java:137)
	at org.openide.util.actions.ActionInvoker$1.run(ActionInvoker.java:92)
...

TDB07: Avoid runtime annotations where there is a way to do the same thing with plain Java code.

TDB08: What is @InsideEdt for?  I see some code that applies it, but none that actually consumes it.

TDB09: SwingUtilities.invokeAndWait() should basically never be used - there are too many foreign locks involved.  If you really need to block while doing something in the EDT, use invokeLater() and a CountDownLatch(1) - you don't necessarily know what locks the calling code is holding and invokeLater() will take publicly visible locks in AWT.

Let me try to paint the bigger picture that represents my view and experience.

I largely write Java GUI code that runs as Jini, ServiceUI clients.  I have a client, desktop environment, that uses Jini lookup to find services, and then uses the service meta data to provide the user a chance to open that service's UI and interact with it.  Multiple UIs can be active simultaneously.  All services have sets of codebase URLS that are used to create the services context classloader.

In this environment, a security manager is always active, and an appropriate Policy is in effect, limiting the client, through the security manager as well as potentially though a Subject, if the downloaded GUI uses that form of authentication and then authorization management.

Thus, all GUI code that uses EDT driven actions, must take into account, that a specific context class loader is active as well as the fact that a specific Subject may be active as well.  So any background threads much be presented this environment to run in.

So, when I write actions, I have to deal with these issues as the facade that must always be maintained.  Thus, I have frameworks to do this so I don't have to do it over and over.  This stuff represents some extractions from what I've been using for years to try and help push the Netbeans community towards an API that will get developer to stop using the EDT for computational task so that anything beyond calls to swing APIS is done in background threads.  This can seem painful to do, unless there is a framework that makes it simple.

I read the through the discussion about using capabilities.  I think that practically, it's at the wrong level for what I am trying to do.  Historically, I view alot of the netbeans APIS actually trying to solve everything for everybody by exposing too many low level details in high level APIs.  For example, the bit about "e.t. for completion" in that discussion is not appropriate for consideration of this framework.  I don't want to expose "Swing" anything, only separation of context between "GUI changes" and "background work".

The code here is driven by my desire to get thread context management, in Swing applications RIGHT, once and for all.  Swingworker, by itself is not an answer (as you suggest not being inspired by it), at all because I can't deal with the issues described above.  But it's concepts are, for me, a great model for separating out the thread contexts that are most often unknown by users of Swing.

Thus, ControlProcessor is a stab at taking all the things that I've been doing and throwing in some customization points.

ControlProcessor is the thing that processes events, controlling thread context and making it plain what things should be in the EDT and what things should not.

The InsideEDT and OutsideEDT are meant for application verification.  It allows analysis of flow of control so that an InsideEDT method can not call out to an OutsideEDT method which would cause EDT hangs.

ContextAccess provides the factory for ControlContext, which is the SPI for how applications can setup the way that the EDT events and background thread context is setup.  ControlContextProviderImpl uses 
DefaultControlContext, which is what I need for my type of Jini based ServiceUI applications.

So, my ControllableAction class, which is just a simple example uses ContextAccess.getContext() to get an instance of ControlAccess to then use for dispatching different types of work to run in specifically appointed contexts.

Now, the final bits...  Ultimately, I am trying to get to the point of being able to run Jini ServiceUI bits inside of the IDE so that as I design Jini services and client UIs, I can test them, in place.  To do this, I must correctly manage the context class loader to download the client codebase jars, as well as manage the SecurityManager issues.

It is completely because of the fact that this type of API is not being used inside of netbeans that I can't successfully make Jini clients work.  The IDE asserts a classloader model and a security model which is completely incompatible without some really painful steps.

If something like this was in place, then I could load "DefaultControlContext" in as a module that would allow the lookup() call in ControlContext to find the context that I needed to be asserted.

Then, I could expand and extend the IDE using Jini clients talking to Jini services...

Whew...

I ended up spending the afternoon prototyping something like what I described above.

I think it would be possible to do a fairly simple and API for background tasks that does what you want, and also allows you to handle classloader and security issues.  Given the following entry points, where would you need to intervene and set the classloader?
 - When a background task is created?
 - When a background task is run?
 - When a background task publishes something on the EDT?
 - Somewhere else?

BTW, if OutsideEDT and InsideEDT are used for static verification, why is their retention RUNTIME?

Have a look at the API defined in api.progress.transactional and progress.transactional, which I just committed into main/.

It certainly encouraging doing I/O and other operations in the background, by handling all of the threading details of doing so, and making it dead-simple to chain together multiple operations, and have them transparently become steps in a progress bar (if desired), publish onto the event thread synchronously or asynchronously.

What is probably interesting for this issue is the set of callbacks available, which was written with this issue in mind.  Specifically, a separate module implements the code which actually handles launching a background task.  A task is divided into transactions, each of which has input and output arguments and can be chained together.  For each background task, there is an instance of a UI class, which is a set of callbacks before and after an entire transaction is run, before and after any individual sub-transaction is run, before pushing a runnable on the event thread, before invoking that runnable on the event thread and after invoking that runnable on the event thread:

public void onBeginSeries(Transaction<?,?> series, TransactionController controller);
public void onEndSeries(Transaction<?,?> series, TransactionController controller);
protected void onBeginTransaction (Transaction<?,?> transaction);
protected void onEndTransaction(Transaction<?,?> transaction);
protected void onBeginRollback(Transaction<?,?> transaction);
protected void onEndRollback(Transaction<?,?> transaction);
protected void onFailure (Transaction<?,?> transaction, Throwable t, boolean isRollback);
protected void onBeforePublish(Transaction<?,?> transaction, Runnable toPublish, boolean synchronous);
protected void onPublish (Transaction<?,?> transaction, Runnable toPublish, boolean synchronous);
protected void onAfterPublish(Transaction<?,?> transaction, Runnable toPublish, boolean synchronous);
public void onBeginRollbackSeries (Transaction<?,?> transaction, TransactionController controller);
public void onEndRollbackSeries (Transaction<?,?> transaction);

I believe this provides all of the entry points you would need to set up whatever plumbing you need, without users who don't need it needing to know about it or deal with it.

There is a module that provides the infrastructure for doing all of the plumbing - handling the threading, etc.  It is a factory for UI instances for each background task.

What you would do is simply place your own implementation of TransactionHandler into the default lookup, so that yours is found by default.  It can then look up the actual default one, intercept all of the calls above (including code that initially sets up a background task), do whatever set-up you need to do, and then delegate to the original to actually show the UI, run things, etc.

While transactions are nested within each other, and wrapper each other at times, TransactionHandler.getContents(Transaction) will always give you a List of 1 or more caller-provided transactions;  Transaction also has Class<ArgType> argType() and Class<ResultType> resultType().  For transactions that need special handling where you might need additional metadata, it would be simple enough for you to create a Transaction subclass, and simply look for that in any Transaction passed to the UI callback.

Does this more or less make sense?

I have created a branch transactional-progress-175342 in http://hg.netbeans.org/prototypes/
please make the API ready for review there.

Y01 Provide usecases in arch.xml (I marked the place with XXX).

Where does this stand now.  Do I need to download the branch and built it to take a look at these changes?

FWIW, I just moved it out of the graveyard and into contrib/api.progress.transactional and contrib/progress.transactional - which should make it at least possible to play with it without devoting an hour just to check it out and do a build.