René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Commands become a DAG: split and join","articleBody":"Right now, our commands construct a linear chain of pipes. It would be nice if we could do arbitrary DAGs, something along the lines of:\r\n\r\n```ocaml\r\nval split : cmd -\u003e cmd * cmd\r\n(** [split] a command into two identical commands that have the same output *)\r\n\r\nval as_fifo : cmd -\u003e string\r\n(** take a command and turn it into a named pipe. Similar to \u003c( ... ) in bash. *)\r\n```\r\n\r\nThis could be used like so:\r\n\r\n```ocaml\r\nlet log1, log2 = process \"tail\" [ \"-f\" ; \"log\" ] |\u003e split in \r\nprocess \"cat\" [as_fifo (log1 |. shuf); as_fifo (log2 |. sort)] |\u003e run\r\n```\r\n\r\n\r\nThe named pipes would have to be cleaned up when the Feather.run exits and `Stdlib.at_exit`.\r\n\r\nI looked into this a bit and the most annoying part is that neither `Base` nor `Stdlib` implement a way to get the *name* of a tempfile without actually creating the tempfile. (We want to mkfifo ourselves.) So we'll have to implement our own random tempfile function.\r\n\r\n","author":{"url":"https://github.com/charlesetc","@type":"Person","name":"charlesetc"},"datePublished":"2021-08-18T05:23:33.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/21/feather/issues/21"}