Sunday 26 January 2014

DWS Mandelbrot Explorer Mark II, and random notes about FireMonkey and threads

The DWS Mandelbrot Explorer, which renders tiles generated by Eric Grange's tile server, has been updated.

The following is a braindump of information about the app, about FireMonkey, about threading, and about how they all interrelate. I think some points will be interesting to you.

Miscellaneous things I've learned

  • It turns out that you cannot reliably use two FireMonkey Direct2D TBitmaps in two different threads at the same time. The code had a background thread to download the fractal data and create a tile bitmap, which was then passed to the main thread to draw. (Only one thread accessed each TBitmap at a time, but several threads used independent TBitmaps concurrently.) Sometimes this results in silent failures to update the texture object backing the TBitmap data. The DirectX methods CopyResource and DrawBitmap, and possibly others, can fail.
    I spent a long time investigating this, seeing what I might be able to do to hack in enough thread-awareness into the library that using two theoretically-independent bitmaps at the same time would work. The code is strongly designed around shared objects, used for all operations. I've looked into:
    • Direct3D factories and using the ID2D1MultiThread interface to synchronize (only Windows 7 and up though);
    • the (different) ID3D10Multithread interface for synchronization, which works brilliantly right up until it deadlocks;
    • surface sharing between APIs;
    • DXGI shared surfaces;
    • per-thread instances of render targets and textures;
    • hand-rolled synchronization around specific areas; 
    • ...you name it. Several things have been almost successful but nothing is reliable, and the more I read, especially when there are caveats about certain functionality not working on Vista but only on 7, or only on hardware and not on WARP, the more I understand why it makes sense for the FireMonkey Direct2D canvas implementation not to have even tried to implement it.
      Because of this, there's a lot more processing in the app's main thread, generating the bitmaps as well as just drawing everything onscreen. This is not ideal. It seems FMX apps will unfortunately have to stay away from second-thread graphics processing but this is due to the underlying graphics libraries, specifically here Direct2D. Ie, it's not really FireMonkey's fault. If you really need to, you can use a specific graphics library in your other threads, just don't use lots of plain TCanvases and TBitmaps and expect it to work - keep them in one thread. Graphics32 and VPR might be worth investigating.
  • Also, hacking thread support into a non-threaded library you're not overly familiar with is a hard task. Just saying. Fun though.
  • There seems to be no cross-platform Delphi RTL equivalent of WaitForMultipleObjects. The TEvent class wraps an event, but what happens when you want to wait for two of them? This would make a good open source class, I think (a list of events, with wait methods?) but right now this app's code re-uses one event for several things making it a bit complicated.
  • If you want to interrupt a TIdHTTP downloading (via blocking call TidHTTP.Get), call TIdHTTP.Disconnect. The object seems to be reusable afterwards for a subsequent Get call. I use Disconnect to achieve fast termination of thread downloading when I want to stop the background thread objects and need to wait on them, so need them to stop quickly.
  • The Quartz canvas on OSX is noticeably slower than Direct2D.  I think, without measuring, it might even be slower than GDI+. I am curious why this is and if anyone else has seen the same thing in their FireMonkey apps.
  • TImageControl on OSX does not render correctly when you write to its Bitmap. I traced through the code and am not sure what it's doing - the code looks valid - but I can state that after drawing onto the Bitmap what was displayed onscreen, a blank solid color, was wrong. Since I just needed a canvas to draw on and a control to give mouse events, I ended up just using a TPanel.
  • A FireMonkey TForm is missing some seemingly obvious events: there is no OnClick or OnDblClick.  I shouldn't need a client-aligned panel to give mouse click events, but I did.
  • The look of FireMonkey on Windows has improved greatly since XE2, and is quite close to how native Windows controls / the VCL looks. The following image (click to expand) has exactly the same controls placed in the same position with the same dimensions on a XE2 FMX form, a XE4 FMX form and VCL FMX form.  The XE2 one doesn't look very native; the XE4 one is quite similar to the VCL.

    In some cases I think FireMonkey is better. Look how the TTrackBar's edges align nicely in FireMonkey, for example, but don't in the genuine native control - something that bugs me every time I use the real one.

Notes about the app itself

FireMonkey in practice

  • The point of the app was a write a non-trivial FireMonkey app and see, in practice, what issues arose. I can confidently state I have learned a lot about FireMonkey building this app. That was the goal.
    Has there been anything particularly bad? I don't think so. There were three areas:
    • Bugs: none serious. Graphics performance is easily fixable.
    • Cross-platform: some code is slightly less clear than it could be, because I've stuck to using the cross-platform RTL and FMX only.  (For example, had I been able to drop back to using WaitForMultipleObjects(Ex) some code might have been clearer.)
    • Framework: FireMonkey is different to the VCL, and I limited myself to sticking to it and the Delphi RTL / libraries only rather than using, say, Windows API code. This is no different to learning and using any other framework: the same problems occur learning the .Net libraries, Cocoa, etc.
  • On the whole, FireMonkey is a good framework the details of which you have to learn, like any other. It's backed up by the Delphi RTL which is fairly comprehensive. Put it this way, if you can write a VCL app using Delphi, you can write a FireMonkey one.

Other

  • Finally, thanks to François Piette, who submitted some changes to port from XE2 to XE5. I integrated his changes which improved the code - especially bitmap generation from the downloaded data - greatly. Thanks François!
  • Download version 1.1 or find the source code here. Windows 32, 64 and OSX 32.

Next up will be something completely different.

11 comments:

  1. Thank you for your blog posts on the FMX canvas stuff. I have found it very interesting as I have been wrestling with FMX canvas issues too.

    Interesting you encountered the background thread to bitmap problem as well. When I was writng my SVG library, on mobile devices, it is consistently *unreliable* and I had to give up. In fact, any threading I was writing (even to load from file) was still unreliable and I removed threading for mobile entirely. I just assumed it was the mobile ARM versions and didn't investigate further. Since I only have a Win7 computer to test on, I hadn't realized it was going to gack on Vista as well. What was your experience with MacOS? It seems unreliable as well but I find the FMX MacOS applications that way threaded or not. Like you, I think the FMX is much slower on MacOS. I would love to hear you expand more on this whole issue of FMX and threads.

    ReplyDelete
    Replies
    1. Thanks Tom - I'm glad you find it interesting.

      Re graphics and threading, I think two things:
      a) It's completely valid to expect to be able to do graphic operations in a background thread, as in the producer-consumer model this app used. So not being able to use independent TBitmaps in threads because it turns out they're not independent after all is a shame.
      b) After digging into the Direct2D code, I can see why they're not independent. But this is implementation-specific; I didn't run into any problems using the GDI+ canvas, for example. I think our expectations are high because we can use VCL TBitmaps, TCanvases etc (wrappers around GDI) in different threads with almost no worries.

      What threading issues did you encounter? From what you wrote it looks like it might be more than just issues while drawing graphics - you had file code as well that was causing problems?

      MacOS FireMonkey applications are definitely not as slick or fast as Windows FireMonkey applications. I hope to see this improved by Embarcadero - I think some optimising and speedups would improve the platform greatly.

      Delete
    2. Hi Dave,
      a) I agree. Graphics in a background thread should work. I am rendering SVGs and with complex SVGs these can take a long time. Having that option is useful.

      To be fair to Embarcadero/FMX, it seems like I am always squashing bugs in my multi-threaded code, but on mobile it was failing spectacularly and everywhere. And I find the remote debugging to be so bad that it is hard to figure out what is going on.

      Not only wouldn't the bitmap rendering work, but even loading an XML file in a thread wouldn't. There doesn't seem to be a CoInitialize/UnCoInitialize call to use like you should do with the MS XML parser on Windows multi-threading. I would think that this must work as I expect the XML REST stuff must be using threads. But since most of my speed issues were on the graphics side and this was not a main feature, at that point I shelved it for later investigation.

      Delete
  2. Interesting article Dave - could you provide more info on the settings that produced the middle (XE4/FMX) dialog? I've recently started out on my first major FMX app. I like the FMX architecture, but the text quality is a major problem. I can get reasonable quality if I use GDI+ (although still nowhere near the quality of WPF) but the D2D is unusable. Of course, with GDI+ I get the dancing text.

    Menu items are a particular problem - it's a struggle to identify which items are enabled/disabled. How are they on your system?

    ReplyDelete
    Replies
    1. The XE4 FMX app uses the default settings - I made a new FMX app in XE2, then opened it in XE4 to take that screenshot.

      Menu items: I experimented with main menus (which are fine and render just like Windows default menus, which it's possible they are) and popup menus. Popup menus render differently to main menus and the disabled item text is darker than on a main menu, so it is harder to distinguish. Strange.

      I saw little difference in text rendering between GDI+ and D2D in this (very simple) app, even adding a memo with several lines of text. Character spacing was often slightly wider. There was no noticeable antialiasing quality difference. What did you see in your app?

      In general I'd recommend using D2D, including for text - several major Windows components, like the new IE rendering engine as far as I know, use it.

      Delete
    2. After looking a bit more closely:
      - TMainMenu: appears to use the real Windows menu code (and OSX) to draw, so anything there will render perfectly.
      - TMenuBar and TPopupMenu: render using FMX styles, which do not mimic Windows menus very well. The font isn't right and text colour for disabled items isn't good either, as you noted.

      I hadn't looked at menus at all before this, so apologies for the slightly ad-hoc replies here.

      I'd say this is a good opportunity to patch the FMX menu styles and release a better version :)

      Delete
    3. With D2D the text is slightly fuzzy and is considerably lighter than WPF. I use WPF as the "gold standard" since it's what FMX should be able to achieve given their vector nature. It's interesting to trawl through the history of WPF text rendering on the web and see that it had similar problems to FMX in the early days with fuzzy text.

      Delete
  3. Thanks for checking that out - I'm using TMenuBar. Interesting re your XE4 app - what OS and screen resolution are you using?

    ReplyDelete
    Replies
    1. It's Windows 7 Pro at 1680x1050, with default font sizes. Nothing unusual in the font or screen setup.

      Do you have any links to a history of WPF's text rendering, and (even better) a link to anything about what their solution was? Maybe we can patch FMX to do the same thing.

      Delete
    2. I'd been trawling around looking for potential patches that I could apply to FMX, but it really isn't something I know enough about. One interesting link is this one http://www.hanselman.com/blog/WPFAndTextBlurrinessNowWithCompleteClarity.aspx where criticisms of an apps font are mentioned. There are lots of articles where this is mentioned - here's another http://www.actiprosoftware.com/support/kb/article/20003/ways-to-fix-wpf-fuzzy-text-rendering-issues. I'm optimistic this will get fixed - WPF looks great on my laptop so it is do-able. Other links are http://stackoverflow.com/questions/190344/wpf-blurry-fonts-problem-solutions and http://10rem.net/blog/2010/06/06/wpf-and-silverlight-choose-your-fonts-and-text-rendering-options-wisely.

      Delete
  4. FYI, calling TIdHTTP.Disconnect() is not the only way to abort a download in progress. You can also use the TIdHTTP.OnWork... events to raise an exception when needed, such as by calling SysUtils.Abort(). You should have a try/except block around the TIdHTTP.Get() call anyway, so you can handle an abortive exception from there.

    ReplyDelete