René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Newer API","articleBody":"Our current API is a bit clunky and doesn't allow supporting all of the proposed features we wanted.\r\n\r\nSo, here's some newer endpoints I propose:\r\n\r\n\r\n### Executing a basic subprocess\r\n```c++\r\n/**\r\n * Execute a subprocess and optionally call a function per line of stdout.\r\n * @param commandPath   - the path of the executable to execute, e.g. \"/bin/cat\"\r\n * @param commandArgs   - the extra arguments for an executable e.g. {\"argument 1\", \"henlo\"}\r\n * @param stdinInput    - a list of inputs that will be piped into the processes' stdin\r\n * @param lambda        - a function that is called with every line from the executed process (default NOP function)\r\n * @param env           - a list of environment variables that the process will execute with (default nothing)\r\n */\r\nint execute(const std::string\u0026 commandPath, const std::vector\u003cstd::string\u003e\u0026 commandArgs, const std::vector\u003cstd::string\u003e\u0026 stdinInput, const std::function\u003cvoid(std::string)\u003e\u0026 lambda = [](std::string){}, const std::vector\u003cstd::string\u003e\u0026 env = {});\r\n```\r\nAlso, another `execute` function will exist, except that instead of taking in a std::vector to supply its stdin, it takes two input iterators (the start and end of a range). It's signature would look like this:\r\n```c++\r\ntemplate\u003cclass InputIt\u003e\r\nint execute(const std::string\u0026 commandPath, const std::vector\u003cstd::string\u003e\u0026 commandArgs, InputIt stdioBegin, InputIt stdioEnd, const std::function\u003cvoid(std::string)\u003e\u0026 lambda = [](std::string){}, const std::vector\u003cstd::string\u003e\u0026 env)\r\n```\r\nWhere the InputIt would require typical iterator methods implemented (similar to InputIterator defined here: https://en.cppreference.com/w/cpp/named_req/InputIterator).\r\n\r\n### Retrieving subprocess output\r\nSimilar to the `execute` function's parameters, the `check_output` function (name TBD, this name is just copied from the python3 subprocess library, we should check if they have better names (or there's internal conflict about how shit that name is)) simply returns a vector containing all lines of stdout.\r\n\r\n```c++\r\nstd::vector\u003cstd::string\u003e check_output(const std::string\u0026 commandPath, const std::vector\u003cstd::string\u003e\u0026 commandArgs, const std::vector\u003cstd::string\u003e\u0026 stdioInput, const std::vector\u003cstd::string\u003e\u0026 env= {})\r\ntemplate\u003cclass InputIt\u003e\r\nstd::vector\u003cstd::string\u003e check_output(const std::string\u0026 commandPath, const std::vector\u003cstd::string\u003e\u0026 commandArgs, InputIt stdioBegin, InputIt stdioEnd, const std::vector\u003cstd::string\u003e\u0026 env = {});\r\n```\r\n\r\nInput iterator is similar to the above restrictions in the `execute` function.\r\n\r\n### Interactive input/output\r\nTo enable dynamically setting what should be fed into the program based on the output, we introduce a newer ProcessStream class. This class has only a few methods available, but allows feeding a string into the process, and checking whether a line is available to read (there may not be one always available for each stdin, consider grep when the line doesn't match) and also extracting it. \r\n\r\nI imagine it's use will be something along these lines:\r\n```c++\r\n// ctor\r\nProcessStream(const std::string\u0026 commandPath, const std::vector\u003cstd::string\u003e\u0026 commandArgs, const std::vector\u003cstd::string\u003e\u0026 env);\r\n\r\nProcessStream ps(\"/bin/cat\", {});\r\n\r\nps \u003c\u003c \"henlo world\\n\";\r\n\r\nif (ps.ready()) {\r\n    std::string output;\r\n    output \u003c\u003c ps;\r\n    std::cout \u003c\u003c output;\r\n}\r\n```\r\n\r\nThere will naturally be non-operator overload equivalents to those endpoints (most likely something like `.write(std::string)` and `std::string .read(timeout)`.\r\n\r\nWe require a timeout, because it's not really possible to check if there *is* a line available, without some kind of call to select (I think!).\r\n\r\n### Daemon spawning\r\nPeople seem to like to spawn and forget processes (especially in some kind of management application), so providing a method to disown and mark them as daemons makes a whole lotta sense. I haven't written daemon code too often (it's been a couple years since I have I reckon), and that was in pure C. As this library *has to use C* to implement these features it's certainly doable, but would require different behaviour per OS, because this I imagine heavily differs between OSes. The problem is, I don't really know what a nice interface to doing so would be.\r\n\r\nAny suggestions would be much appreciated.\r\n\r\n### Error Handling\r\nAs we're dealing with external applications, they may throw at any time and inconsistently (either due to the application, the OS/user killing them, or bugs), we need some kind of error handling.\r\n\r\nCurrently, the above APIs proposed return an error status, but I feel like this may not be very idiomatic, and that exceptions should be used instead. I suppose this requires a read up on C++ philosophy before being able to make a decision, but I'm thinking of mirroring how Python's subprocess library does it, which is pretty much to throw an exception whenever anything goes wrong, including non-zero return statuses.\r\n\r\n\r\n### Asynchronous Calls\r\nWhether the user should be responsible for running this stuff in the background is again another philosophical question to ask, but as this is a library aimed at dumb-dumbs, its worth consideration if this is something we should hold their hand on. Currently, we provide a method to spawn a subprocess asynchronously, and return a `std::future` of the return code. Naturally, if we change the error handling to be a different strategy, this may not make sense, but I can't imagine asynchronous informing the main process about the status via exceptions can't be neat, so using futures on the return value seems like a good idea to cover this.\r\n\r\n### Final remarks\r\nIn my mind, these cover the current TODOs I propose in the current README, so it seems like a good direction to head. The only thing that I don't know and could cause problems in these interfaces is how Windows deals with processes. Some of these things may not be possible.\r\n\r\nAny comments are appreciated (and likely only Cameron is gonna comment, but I'll try and hook some people to comment).\r\n\r\nCheers","author":{"url":"https://github.com/pnappa","@type":"Person","name":"pnappa"},"datePublished":"2018-11-17T10:54:21.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":22},"url":"https://github.com/3/subprocesscpp/issues/3"}