René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Extremely low speed of ReadBufferDataHandle","articleBody":"I tried to use ReadBufferDataHandle to speed up parsing TIFF, instead of simple FileHandle in my analog of TiffParser. And I was very surprised that the speed did not increase.\r\n\r\nThen I've created the following very simple test, which reads first 250000 of 4-byte integer values from some test file:\r\n\r\n```java\r\npublic class ReadArraySpeed {\r\n    public static void main(String[] args) throws IOException, FormatException {\r\n        if (args.length \u003c 1) {\r\n            System.out.println(\"Usage:\");\r\n            System.out.println(\"    \" + ReadArraySpeed.class.getName() + \" any_file\");\r\n            return;\r\n        }\r\n\r\n        final Location location = new FileLocation(new File(args[0]));\r\n        for (int test = 1; test \u003c= 5; test++) { // - helps to warm JVM\r\n            System.out.printf(\"Test #%d...%n\", test);\r\n            try (Context context = new Context();\r\n                DataHandle\u003cLocation\u003e in = context.getService(DataHandleService.class).create(location);\r\n                 DataHandle\u003cLocation\u003e inBuffer = context.getService(DataHandleService.class).readBuffer(location)) {\r\n                final int elementSize = 4;\r\n                final int size = (int) (Math.min(in.length(), 1_000_000) / elementSize);\r\n                int[] a = new int[size];\r\n                int[] b = new int[size];\r\n                long t1 = System.nanoTime();\r\n                in.seek(0);\r\n                for (int k = 0; k \u003c a.length; k++) {\r\n                    a[k] = in.readInt();\r\n                }\r\n                long t2 = System.nanoTime();\r\n                inBuffer.seek(0);\r\n                for (int k = 0; k \u003c b.length; k++) {\r\n                    b[k] = inBuffer.readInt();\r\n                }\r\n                long t3 = System.nanoTime();\r\n                System.out.printf(Locale.US, \"Reading %d 32-bit integers, %s: %.3f ms, %.3f MB/s%n\",\r\n                        size, in.getClass(),\r\n                        (t2 - t1) * 1e-6, (size * elementSize) / 1048576.0 / ((t2 - t1) * 1e-9));\r\n                System.out.printf(Locale.US, \"Reading %d 32-bit integers, %s: %.3f ms, %.3f MB/s%n\",\r\n                        size, inBuffer.getClass(),\r\n                        (t3 - t2) * 1e-6, (size * elementSize) / 1048576.0 / ((t3 - t2) * 1e-9));\r\n            }\r\n        }\r\n    }\r\n}\r\n```\r\nOn my computer, results are following:\r\nTest #1...\r\nReading 250000 32-bit integers, class org.scijava.io.handle.FileHandle: 317.620 ms, 3.003 MB/s\r\nReading 250000 32-bit integers, class org.scijava.io.handle.ReadBufferDataHandle: 2609.762 ms, 0.365 MB/s\r\nTest #2...\r\nReading 250000 32-bit integers, class org.scijava.io.handle.FileHandle: 336.619 ms, 2.833 MB/s\r\nReading 250000 32-bit integers, class org.scijava.io.handle.ReadBufferDataHandle: 2769.079 ms, 0.344 MB/s\r\nTest #3...\r\nReading 250000 32-bit integers, class org.scijava.io.handle.FileHandle: 302.576 ms, 3.152 MB/s\r\nReading 250000 32-bit integers, class org.scijava.io.handle.ReadBufferDataHandle: 2784.298 ms, 0.343 MB/s\r\nTest #4...\r\nReading 250000 32-bit integers, class org.scijava.io.handle.FileHandle: 307.343 ms, 3.103 MB/s\r\nReading 250000 32-bit integers, class org.scijava.io.handle.ReadBufferDataHandle: 2815.823 ms, 0.339 MB/s\r\nTest #5...\r\nReading 250000 32-bit integers, class org.scijava.io.handle.FileHandle: 317.017 ms, 3.008 MB/s\r\nReading 250000 32-bit integers, class org.scijava.io.handle.ReadBufferDataHandle: 3378.953 ms, 0.282 MB/s\r\n\r\nYou see that ReadBufferDataHandle works... not only not faster, but much slower than the  simple DataHandle.\r\nDebugger shows that it performs a lot of calls of methods of low-level RandomAccessFile class, like exists(), length(). Maybe it is the reason.\r\n\r\nCould you cleanup ReadBufferDataHandle to make it really quick while the simple sequential reading?\r\n\r\nFor comparison, if you will replace \"readInt()\" call with \"readByte()\", you will see the expected result: ReadBufferDataHandle works much faster. This method is implemented in much more simple way and is really efficient. But reading single byte is a rare case, usually we need to read some other types or short buffers.\r\n\r\n\r\n","author":{"url":"https://github.com/Daniel-Alievsky","@type":"Person","name":"Daniel-Alievsky"},"datePublished":"2023-08-04T10:49:24.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":6},"url":"https://github.com/467/scijava-common/issues/467"}