René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Allocation of 360434219 exceeds 10% of free system memory.","articleBody":"\u003cem\u003ePlease make sure that this is a bug. As per our [GitHub Policy](https://github.com/tensorflow/tensorflow/blob/master/ISSUES.md), we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template\u003c/em\u003e\r\n\r\n**System information**\r\n- Have I written custom code (as opposed to using a stock example script provided in TensorFlow):\r\n- OS Platform and Distribution (e.g., Linux Ubuntu 16.04 x86\\_64): **Linux x86_64 in a Docker cointainer**\r\n- TensorFlow installed from (source or binary):\r\n- TensorFlow version (use command below): **1.0.0-RC.2**\r\n- Java version (i.e., the output of `java -version`): **openjdk version \"21.0.4\"**\r\n- Java command line flags (e.g., GC parameters):\r\n- Python version (if transferring a model trained in Python): **3.9**\r\n- Bazel version (if compiling from source):\r\n- GCC/Compiler version (if compiling from source):\r\n- CUDA/cuDNN version:\r\n- GPU model and memory:\r\n\r\n**Describe the current behavior**\r\nI am using TensorFlow in a Spring Boot application, which exposes an endpoint for NER processing. The TensorFlow model is trained in Python and loaded into the Java application for inference.\r\n\r\nTo optimize performance, I initialize the TensorFlow session once during application startup using a @PostConstruct method and store it in a private field:\r\n\r\n```\r\nprivate Session session;\r\n\r\n@PostConstruct\r\nprivate void initialize() throws IOException {\r\n    byte[] bytes = Files.readAllBytes(Paths.get(\"/path/to/model/\"));\r\n    Graph graph = new Graph();\r\n    graph.importGraphDef(GraphDef.parseFrom(bytes), \"PREFIX\");\r\n    session = new Session(graph);\r\n}\r\n```\r\n\r\nThe session is reused in a public method for running predictions:\r\n\r\n```\r\npublic Result predict(String input) {\r\n    try (Tensor textTensor = Tensor.of(TINT32.class, ...);\r\n         Result result = session.runner()\r\n                                .feed(\"otherOperationName\", textTensor)\r\n                                .fetch(\"operationName\")\r\n                                .run()) {\r\n        // Process the result here\r\n    }\r\n}\r\n\r\n```\r\n\r\nDuring performance testing, I monitored the heap memory and found no significant issues. However, when the application runs in a Docker container, it crashes after a while, regardless of the memory allocated to the container (even with 120GB of memory). The following warning appears in the logs before the crash:\r\n\r\n`W external/local_tsl//framework/cpu_allocator_impl.cc:83] Allocation of 34891293 exceeds 10% of free system memory.`\r\n\r\nIs it possible that the memory leak is caused by the session being stored in a private field and never explicitly closed, even though all tensors and intermediate results are properly managed (closed) in the predict method?\r\n\r\n**Describe the expected behavior**\r\nThe application should not exhibit memory leaks or crashes when deployed in a Docker container, regardless of memory allocation.\r\n","author":{"url":"https://github.com/icrecescu","@type":"Person","name":"icrecescu"},"datePublished":"2024-11-25T06:25:47.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":12},"url":"https://github.com/588/java/issues/588"}