René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add support for pre-multiplied alpha","articleBody":"At the bottom of this pull request (https://github.com/processing/processing/pull/2601) @benfry mentioned opening an issue about implementing pre-multiplied alpha, which I don't think was ever done? I certainly dropped the ball on following up.  It would be great if an official image mode for this was added (I'm currently sub-classing the GL renderer to achieve this).  Otherwise multiple layering (as well as convolution, etc) don't work correctly.\n\nFor reference, a long and admittedly opinionated extract of an email sent to @JakubValtar last year with a suggestion for handling this internally in the OpenGL renderer - feel free to TL;DR, although the NVIDIA link is worth a read.\n\n---\n\nOK, you asked me how Processing might use pre-multiplied alpha without\nbreaking the API.  So to start, a possibly controversial statement -\nblending in Processing is completely broken.  It was one of the\nreasons I stopped using Processing the first time around (~2006) - I\nlike the API but in my opinion there are some issues with the\nimplementation.  Praxis LIVE v1 deliberately made use of aspects of\nthe Processing API, but re-implemented on top of a custom software and\nOpenGL renderer.  Fast forward to a year ago, and I decided to stop\ntrying to re-implement the API - the new Processing OpenGL is too good\nto miss!  So Praxis LIVE v2 now embeds Processing as a library -\nhowever, there's no way I'm breaking blending in the process - luckily\nit isn't that difficult to work around.\n\nThere are many sources around about pre-multipled alpha.  I found this\npage recently looking into GL blend modes\n(http://www.slideshare.net/Mark_Kilgard/blend-modes-for-opengl) - it's\nabout the new NVIDIA blend mode extensions - note that slide 16 talks\nabout the fact that blend modes assume pre-multiplied alpha.  It\nmentions one of the key issues (one that affects me) - drawing into\nsurfaces with alpha channels - sometimes you can get away with it if\nthe destination is opaque.  He also mentions the Porter-Duff model and\nhow it relies on pre-multiplied alpha - interesting that the default\nblending in the Java2D renderer works fine - because the JRE\nAlphaComposite  pre-multiplies, blends and un-premultiplies.\n\nIncidentally, that slide also touches on another issue -\npre-multiplied alpha is necessary for performing various filtering\neffects (eg. blur) on surfaces with alpha.  I haven't looked at the\nPImage filters but I imagine there might be an issue there (note the\nConvolveOp in the JRE multiplies, filters and unmultiplies).\n\nNow, all that multiplying and unmultiplying is a bit pointless! ;-)\nNot to mention it's not easily achievable with OpenGL.  So, how to use\npre-multiplied alpha without changing the API?\n\nThe important thing in my opinion is separating the external format of\nimages to the format they're stored internally.  Take managed images\nin the Java2D backend - cached textures are not stored in the format\nof the BufferedImage, but the optimum format for rendering.  Just\nbecause the format to the user is un-premultiplied ARGB doesn't mean\nthat the underlying storage has to be.  Processing could easily switch\nto storing the underlying pixel data, in all modes, as pre-multiplied.\nOr, simpler, though not fixing issues elsewhere, it could just store\npre-multiplied data in the OpenGL textures.\n\nThe key things to handle are times when the user is setting colours,\ntimes when the user is retrieving colours, and access to the pixel\narray.\n\nProcessing already does colour calculations to match a colour format\nset by the user and convert into internal data.  What I've done in\nPraxis LIVE's PGraphics sub-class is override the various colorCalc()\nmethods to multiply in the alpha.  That's pretty much all that I'm\nusing right now, but then I don't have 100% API coverage!  It would be\npossible to handle multiply and unpremultiply fairly easily in set()\nand get(), which leaves the pixel array - I'd suggest running a\nun-premultiply \u003c\u003e premultiply during loadPixels() and updatePixels(),\nwith an extra hint to leave it premultiplied and save any hit (of\ncourse RGB images wouldn't have any anyway).\n\nI'm not quite sure if there would be any implications to PShape that\nwould need looking at - I guess I'll find out when I come to\nintegrating support for them! ;-)\n\nThe one aspect of the API that would change would be GLSL - mind you\nI'd assert that's fixing it rather than breaking it - however, it's\nthe one place that might affect the behaviour of user code.\n\nIncidentally, if it's of interest - some alternate software blend\nmodes that I and another person on the Java Gaming forum worked on a\nfew years back - all using pre-multiplied alpha, obviously -\nhttp://www.java-gaming.org/topics/some-new-blendmodes-add-multiply-overlay-etc/24529/view.html\n","author":{"url":"https://github.com/neilcsmith-net","@type":"Person","name":"neilcsmith-net"},"datePublished":"2015-06-16T16:45:52.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":14},"url":"https://github.com/3391/processing/issues/3391"}