René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Develop a way to make Ops usable easily from CompGraph framework","articleBody":"Here are some interfaces we came up with that could deliver what CompGraph needs on the SciJava/ImageJ Ops side:\r\n\r\n\u003cdetails\u003e\u003csummary\u003eBase interface for structured computations:\u003c/summary\u003e\r\n\r\n```java\r\npublic ComputationStructure\u003cS\u003e {\r\n\tdefault S transform(S in) { return in1; } // e.g. identity\r\n\tS requiredInput(S out, S in);\r\n}\r\n\r\npublic ComputationBiStructure\u003cS\u003e {\r\n\tdefault S transform(S in1, S in2) { return in1; } // e.g. identity\r\n\tdefault S requiredInput1(S out, S in1, S in2) { }\r\n\tdefault S requiredInput2(S out, S in1, S in2) { }\r\n}\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\nSciJava/ImageJ Ops need a way to deliver objects implementing the appropriate (unary or binary) base interface. The issue is that a higher-arity op needs to lock down its \"secondary\" parameters (i.e. non-images; things that KNIME would have a user specify in the configuration dialog), to reduce arity down to unary or binary. We could have interfaces like the following to accomplish this:\r\n\r\n\u003cdetails\u003e\u003csummary\u003eStructured, BiStructured, etc.\u003c/summary\u003e\r\n\r\n```java\r\npublic final class Structured\u003cS\u003e {\r\n\tComputationStructure\u003cS\u003e structure();\r\n}\r\n\r\npublic final class BiStructured\u003cS\u003e {\r\n\tComputationBiStructure\u003cS\u003e structure();\r\n}\r\n\r\npublic final class Structured.With1Param\u003cS, P\u003e {\r\n\tComputationStructure\u003cS\u003e structure(P param);\r\n}\r\n\r\npublic final class Structured.With2Params\u003cS, P1, P2\u003e {\r\n\tComputationStructure\u003cS\u003e structure(P1 param1, P2 param2);\r\n}\r\n\r\npublic final class BiStructured.With1Param\u003cS, P\u003e {\r\n\tComputationBiStructure\u003cS\u003e structure(P param);\r\n}\r\n\r\npublic final class BiStructured.With2Params\u003cS, P1, P2\u003e {\r\n\tComputationBiStructure\u003cS\u003e structure(P1 param1, P2 param2);\r\n}\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\nThen for ImageJ Ops, we need to specify the structure `S` as some metadata container. Probably something like this:\r\n\r\n```java\r\npublic class ImageInfo {\r\n\tObject elementType(); // i.e. the ImgLib2 pixel type T\r\n\tInterval interval();\r\n}\r\n```\r\n\r\nHere is an example: gaussian convolution with locked down `double[] sigmas`:\r\n\r\n\u003cdetails\u003e\u003csummary\u003eGaussianConvolveWithSigmas\u003c/summary\u003e\r\n\r\n```java\r\npublic class GaussianConvolveWithSigmas\u003cT extends RealType\u003cT\u003e\u003e implements Computers.Arity2\u003cRAI\u003cT\u003e, double[], RAI\u003cT\u003e\u003e, Structured.With1Param\u003cImageInfo, double[]\u003e {\r\n\tpublic void compute(RAI\u003cT\u003e image, double[] sigmas, RAI\u003cT\u003e output) {\r\n\t\t// ...\r\n\t}\r\n\r\n\tpublic ComputationStructure\u003cImageInfo\u003e structure(final double[] sigmas) {\r\n\t\treturn new ComputationStructure\u003cImageInfo\u003e() {\r\n\t\t\t@Override\r\n\t\t\tpublic ImageInfo transform(final ImageInfo in) {\r\n\t\t\t\t// e.g. identity\r\n\t\t\t\treturn in;\r\n\t\t\t}\r\n\t\t\t@Override\r\n\t\t\tpublic ImageInfo requiredInput(final ImageInfo out, final ImageInfo in) {\r\n\t\t\t\treturn new ImageInfo() {\r\n\t\t\t\t\t@Override\r\n\t\t\t\t\tpublic Object elementType() { return in.elementType(); }\r\n\t\t\t\t\t@Override\r\n\t\t\t\t\tpublic Interval interval() {\r\n\t\t\t\t\t\t// NB: Derive required input interval from output interval + sigmas.\r\n\t\t\t\t\t\treturn requiredInputComputedFromSigmas(out.interval(), sigmas); // Could also rely on in.interval() if needed.\r\n\t\t\t\t\t}\r\n\t\t\t\t}\r\n\t\t\t}\r\n\t\t}\r\n\t}\r\n}\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\nThe `double[] sigmas` state here is implicit in the new `ComputationStructure` anonymous class we define.\r\n\r\nOne question this does not answer is: how to go from an actual input object (e.g., `RandomAccessibleInterval\u003cDoubleType\u003e` to its corresponding structure (`ImageInfo`)? We could use an extensible service/plugin mechanism. Or just keep it simple and rely on the caller to take care of this somehow—in which case maybe `ImageInfo` has constructors that are helpful for this.\r\n\r\nFor now, let's assume we have method `structureOf(inputValue)\" available that returns `ImageInfo`.\r\n\r\nHere is another example that maps to a binary structured computation:\r\n\r\n\u003cdetails\u003e\u003csummary\u003eMatrix multiplication\u003c/summary\u003e\r\n\r\n```java\r\npublic class MatMultiply\u003cT extends RealType\u003cT\u003e\u003e implements Computers.Arity2\u003cRAI\u003cT\u003e, RAI\u003cT\u003e, RAI\u003cT\u003e\u003e, BiStructured\u003cImageInfo\u003e {\r\n\tpublic void compute(RAI\u003cT\u003e mat1, RAI\u003cT\u003e mat2, RAI\u003cT\u003e out) {\r\n\t\t// NB: We can validate the inputs+output structure here! Nice!\r\n\t\tcheckStructure(mat1, mat2, out);\r\n\t\tS s1 = structureOf(mat1), s2 = structureOf(mat2);\r\n\t\tS expectedStructure = structure().transform(s1, s2);\r\n\t\tif (!expectedStructure.equals(structureOf(out))) throw new IllegalArgumentException(\"Output structure not compatible with input structures.\");\r\n\t\t// OR: Better might be to do this the builder, to avoid repeated checking in loops over the same op instance with \"known good\" args.\r\n\r\n\t\t// Finally, do the algorithm here ...\r\n\t}\r\n\r\n\tpublic ComputationBiStructure\u003cImageInfo\u003e structure() {\r\n\t\treturn new ComputationBiStructure\u003cImageInfo\u003e() {\r\n\t\t\t@Override\r\n\t\t\tpublic ImageInfo transform(final ImageInfo in1, final ImageInfo in2) {\r\n\t\t\t\tif (in1.numDimensions() != 2 || in2.numDimensions() != 2) alsoFail(); // also check matching offsets? what is a mat multiply with non-zero offset ???\r\n\t\t\t\tif (!in1.equals(in2)) throw new IllegalArgumentException(\"Input structures do not match.\");\r\n\t\t\t\treturn Interval(in1.interval().dimension(0) + in2.interval().dimension(1)); // too naive -- need to preserve offset\r\n\t\t\t}\r\n\t\t\t@Override\r\n\t\t\tpublic ImageInfo requiredInput(final ImageInfo out, final ImageInfo in) {\r\n\t\t\t\treturn new ImageInfo() {\r\n\t\t\t\t\t@Override\r\n\t\t\t\t\tpublic Object elementType() { return in.elementType(); }\r\n\t\t\t\t\t@Override\r\n\t\t\t\t\tpublic Interval interval() {\r\n\t\t\t\t\t\t// NB: Derive required input interval from output interval + sigmas.\r\n\t\t\t\t\t\t//\r\n\t\t\t\t\t}\r\n\t\t\t\t}\r\n\t\t\t}\r\n\t\t}\r\n\t}\r\n}\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\nNotice that we can check the structure of a computation before trying to do it. We may want to do this outside the actual `compute` method, however, for performance reasons. The builder is a possible place this could happen. Or a new layer could be introduced, perhaps. Regardless, helper methods to validate seem to make sense.\r\n\r\nA key question is whether things behave correctly when computing on non-zero offset tiles/blocks of inputs—since that is the entire point of CompGraph. More thought needed!\r\n\r\n## Next steps\r\n* Consider whether to create composition interfaces for e.g. `Computers.Arity2\u003cI1, I2, O\u003e` + `Structured.With1Param\u003cS, I2(?)\u003e` :arrow_right: `StructuredComputers.Arity2_With1Param\u003cS, I1, I2, O\u003e`\r\n* Ideally, the `structure(...)` implementations should be able to use a lambda in the common case of identity interval transform: we often have same-size input -\u003e output—but the requiredInput computation varies more.\r\n* Abstract base classes can define common behavior for e.g. neighborhood ops. Ideally we make this an interface again with default method(s), rather than abstract class, so lambdas are still usable.\r\n* Consider what to do about `@OpDependency` chaining—can we leverage this somehow in CompGraph? Can we build multiple CompGraph nodes out of composed Ops?\r\n","author":{"url":"https://github.com/ctrueden","@type":"Person","name":"ctrueden"},"datePublished":"2019-09-27T11:48:55.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/37/scijava/issues/37"}