In the course of playing with bug #200748 I noticed a flaw in DesignSupport. The default NB platform version selected when you make a new Maven NBM app is RELEASE701, which will silently ignore the -Dorg.netbeans.core.WindowSystem.designMode=true property. (You can use SNAPSHOT to get a version which interprets it.)
But the same could be true of Ant-based app projects - there is no apparent attempt to verify that the target platform actually has a copy of the o.n.c.windows module that can handle design mode.
Checking the spec version could work. (NbModuleProvider.getDependencyVersion cannot currently be used for this purpose since the Maven impl will not reliably find non-API modules. If the app has been built at least once I think it will work.)
Or you could use a CLI parameter rather than a system property; if the target platform does not support it, there will be a clear error, rather than the current state where the platform starts and there is no indication something is missing.
Integrated into 'main-golden'
User: Jaroslav Tulach <email@example.com>
Log: #201361: Check for presence of org.openide.windows >= 6.45. If present the likeahood of design mode being supported in core.windows is high.