René's URL Explorer Experiment


Title: Build C++ XTensor project as Python package so users can `pip install` without any additional dependencies · Issue #293 · xtensor-stack/xtensor-python · GitHub

Open Graph Title: Build C++ XTensor project as Python package so users can `pip install` without any additional dependencies · Issue #293 · xtensor-stack/xtensor-python

X Title: Build C++ XTensor project as Python package so users can `pip install` without any additional dependencies · Issue #293 · xtensor-stack/xtensor-python

Description: I am following XTensor-python tutorial to build a XTensor C++ project which has Python binding with CMake. The end goal is to create a self-contained Python package that users can install via pip install without requiring any additional ...

Open Graph Description: I am following XTensor-python tutorial to build a XTensor C++ project which has Python binding with CMake. The end goal is to create a self-contained Python package that users can install via pip i...

X Description: I am following XTensor-python tutorial to build a XTensor C++ project which has Python binding with CMake. The end goal is to create a self-contained Python package that users can install via pip i...

Opengraph URL: https://github.com/xtensor-stack/xtensor-python/issues/293

X: @github

direct link

Domain: patch-diff.githubusercontent.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Build C++ XTensor project as Python package so users can `pip install` without any additional dependencies","articleBody":"I am following [XTensor-python](https://github.com/xtensor-stack/xtensor-python) [tutorial](https://xtensor-python.readthedocs.io/en/latest/) to build a XTensor C++ project which has Python binding with CMake. The end goal is to create a self-contained Python package that users can install via `pip install` without requiring **any** additional dependencies. \r\n\r\nThe issue is that the target `mymodule` cannot link to `xtensor::optimize` but only `xtensor`.  \r\n\r\n## Minimal Reproducible Example  \r\nHere is the minimal reproducible example: ([You can follow the instruction here](https://github.com/lionlai1989/xtensor_pybind11_cmake) and reproduce the result)\r\n\r\n#### Project Structure Explanation:\r\n - Project structure:\r\n```shell\r\n.\r\n├── CMakeLists.txt\r\n├── extern\r\n│   ├── pybind11\r\n│   ├── xsimd\r\n│   ├── xtensor\r\n│   ├── xtensor-python\r\n│   └── xtl\r\n├── pyproject.toml\r\n├── setup.py\r\n└── src\r\n    └── main.cpp\r\n```\r\n\r\n - `extern/` folder: `pybind11`, `xtl`, `xtensor`, `xsimd`, and `xtensor-python` are all added as git submodules. The reason is that this project is a self-contained C++/Python project. Users can use only `pip install` to install it without installing anything other things manually.\r\n\r\n - `CMakeLists.txt`: The content is copied and pasted from [xtensor-python tutorial](https://github.com/xtensor-stack/xtensor-python/blob/master/docs/source/examples/readme_example_1/CMakeLists.txt), except `xtensor` packages are not installed and found by `find_package` but with `add_subdirectory` (I will explain the commented code FetchContent later). The reason of not using `find_package` is that I must first install packages so `find_package` will work. But I can't install packages for users on users' computer so packages are added as submodules and added by using `add_subdirectory`.\r\n```\r\ncmake_minimum_required(VERSION 3.22)\r\n\r\nproject(mymodule)\r\n\r\n# Must be submodule. Cannot be installed with pip\r\nadd_subdirectory(extern/pybind11)\r\n\r\n# Use FetchContent\r\n# include(FetchContent)\r\n# # Download and configure xtl\r\n# FetchContent_Declare(xtl\r\n#     GIT_REPOSITORY https://github.com/xtensor-stack/xtl.git\r\n#     GIT_TAG master\r\n#     OVERRIDE_FIND_PACKAGE\r\n# )\r\n# FetchContent_MakeAvailable(xtl)\r\n# # Download and configure xsimd\r\n# FetchContent_Declare(xsimd\r\n#     GIT_REPOSITORY https://github.com/xtensor-stack/xsimd.git\r\n#     GIT_TAG master\r\n#     OVERRIDE_FIND_PACKAGE\r\n# )\r\n# FetchContent_MakeAvailable(xsimd)\r\n# # Download and configure xtensor\r\n# FetchContent_Declare(xtensor\r\n#     GIT_REPOSITORY https://github.com/xtensor-stack/xtensor.git\r\n#     GIT_TAG master\r\n#     OVERRIDE_FIND_PACKAGE\r\n# )\r\n# FetchContent_MakeAvailable(xtensor)\r\n\r\n# Use add_subdirectory\r\nadd_subdirectory(extern/xtl)\r\nadd_subdirectory(extern/xtensor)\r\nadd_subdirectory(extern/xsimd)\r\nadd_subdirectory(extern/xtensor-python)\r\n\r\nfind_package(Python REQUIRED COMPONENTS Interpreter Development NumPy)\r\n\r\npybind11_add_module(mymodule src/main.cpp)\r\ntarget_link_libraries(mymodule PUBLIC xtensor pybind11::module xtensor-python Python::NumPy)\r\n\r\ntarget_compile_definitions(mymodule PRIVATE VERSION_INFO=0.1.0)\r\n```\r\n\r\n- `pyproject.toml`: All required packages to build the system are added here.\r\n```\r\n[build-system]\r\nrequires = [\r\n    \"pip\u003c=23.0\",\r\n    \"setuptools\u003e=62,\u003c=65\",\r\n    \"wheel\",\r\n    \"ninja\",\r\n    \"cmake\u003e=3.22\",\r\n    \"numpy\"\r\n]\r\nbuild-backend = \"setuptools.build_meta\"\r\n\r\n[project]\r\nname = \"mymodule\"\r\nversion = \"0.0.1\"\r\nrequires-python = \"\u003e=3.8\"\r\n\r\ndependencies = [\r\n    \"numpy\"\r\n]\r\n```\r\n\r\n - `setup.py`: The content is from Pybind11/CMake tutorial on [github](https://github.com/pybind/cmake_example/blob/master/setup.py)\r\n```\r\nimport os\r\nimport re\r\nimport subprocess\r\nimport sys\r\nfrom pathlib import Path\r\n\r\nfrom setuptools import Extension, setup\r\nfrom setuptools.command.build_ext import build_ext\r\n\r\n# Convert distutils Windows platform specifiers to CMake -A arguments\r\nPLAT_TO_CMAKE = {\r\n    \"win32\": \"Win32\",\r\n    \"win-amd64\": \"x64\",\r\n    \"win-arm32\": \"ARM\",\r\n    \"win-arm64\": \"ARM64\",\r\n}\r\n\r\n\r\n# A CMakeExtension needs a sourcedir instead of a file list.\r\n# The name must be the _single_ output extension from the CMake build.\r\n# If you need multiple extensions, see scikit-build.\r\nclass CMakeExtension(Extension):\r\n    def __init__(self, name: str, sourcedir: str = \"\") -\u003e None:\r\n        super().__init__(name, sources=[])\r\n        self.sourcedir = os.fspath(Path(sourcedir).resolve())\r\n\r\n\r\nclass CMakeBuild(build_ext):\r\n    def build_extension(self, ext: CMakeExtension) -\u003e None:\r\n        # Must be in this form due to bug in .resolve() only fixed in Python 3.10+\r\n        ext_fullpath = Path.cwd() / self.get_ext_fullpath(ext.name)  # type: ignore[no-untyped-call]\r\n        extdir = ext_fullpath.parent.resolve()\r\n\r\n        # Using this requires trailing slash for auto-detection \u0026 inclusion of\r\n        # auxiliary \"native\" libs\r\n\r\n        debug = int(os.environ.get(\"DEBUG\", 0)) if self.debug is None else self.debug\r\n        cfg = \"Debug\" if debug else \"Release\"\r\n\r\n        # CMake lets you override the generator - we need to check this.\r\n        # Can be set with Conda-Build, for example.\r\n        cmake_generator = os.environ.get(\"CMAKE_GENERATOR\", \"\")\r\n\r\n        # Set Python_EXECUTABLE instead if you use PYBIND11_FINDPYTHON\r\n        # EXAMPLE_VERSION_INFO shows you how to pass a value into the C++ code\r\n        # from Python.\r\n        cmake_args = [\r\n            f\"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY={extdir}{os.sep}\",\r\n            f\"-DPYTHON_EXECUTABLE={sys.executable}\",\r\n            f\"-DCMAKE_BUILD_TYPE={cfg}\",  # not used on MSVC, but no harm\r\n        ]\r\n        build_args = []\r\n        # Adding CMake arguments set as environment variable\r\n        # (needed e.g. to build for ARM OSx on conda-forge)\r\n        if \"CMAKE_ARGS\" in os.environ:\r\n            cmake_args += [item for item in os.environ[\"CMAKE_ARGS\"].split(\" \") if item]\r\n\r\n        # In this example, we pass in the version to C++. You might not need to.\r\n        cmake_args += [f\"-DEXAMPLE_VERSION_INFO={self.distribution.get_version()}\"]  # type: ignore[attr-defined]\r\n\r\n        if self.compiler.compiler_type != \"msvc\":\r\n            # Using Ninja-build since it a) is available as a wheel and b)\r\n            # multithreads automatically. MSVC would require all variables be\r\n            # exported for Ninja to pick it up, which is a little tricky to do.\r\n            # Users can override the generator with CMAKE_GENERATOR in CMake\r\n            # 3.15+.\r\n            if not cmake_generator or cmake_generator == \"Ninja\":\r\n                try:\r\n                    import ninja  # noqa: F401\r\n\r\n                    ninja_executable_path = Path(ninja.BIN_DIR) / \"ninja\"\r\n                    cmake_args += [\r\n                        \"-GNinja\",\r\n                        f\"-DCMAKE_MAKE_PROGRAM:FILEPATH={ninja_executable_path}\",\r\n                    ]\r\n                except ImportError:\r\n                    pass\r\n\r\n        else:\r\n\r\n            # Single config generators are handled \"normally\"\r\n            single_config = any(x in cmake_generator for x in {\"NMake\", \"Ninja\"})\r\n\r\n            # CMake allows an arch-in-generator style for backward compatibility\r\n            contains_arch = any(x in cmake_generator for x in {\"ARM\", \"Win64\"})\r\n\r\n            # Specify the arch if using MSVC generator, but only if it doesn't\r\n            # contain a backward-compatibility arch spec already in the\r\n            # generator name.\r\n            if not single_config and not contains_arch:\r\n                cmake_args += [\"-A\", PLAT_TO_CMAKE[self.plat_name]]\r\n\r\n            # Multi-config generators have a different way to specify configs\r\n            if not single_config:\r\n                cmake_args += [\r\n                    f\"-DCMAKE_LIBRARY_OUTPUT_DIRECTORY_{cfg.upper()}={extdir}\"\r\n                ]\r\n                build_args += [\"--config\", cfg]\r\n\r\n        if sys.platform.startswith(\"darwin\"):\r\n            # Cross-compile support for macOS - respect ARCHFLAGS if set\r\n            archs = re.findall(r\"-arch (\\S+)\", os.environ.get(\"ARCHFLAGS\", \"\"))\r\n            if archs:\r\n                cmake_args += [\"-DCMAKE_OSX_ARCHITECTURES={}\".format(\";\".join(archs))]\r\n\r\n        # Set CMAKE_BUILD_PARALLEL_LEVEL to control the parallel build level\r\n        # across all generators.\r\n        if \"CMAKE_BUILD_PARALLEL_LEVEL\" not in os.environ:\r\n            # self.parallel is a Python 3 only way to set parallel jobs by hand\r\n            # using -j in the build_ext call, not supported by pip or PyPA-build.\r\n            if hasattr(self, \"parallel\") and self.parallel:\r\n                # CMake 3.12+ only.\r\n                build_args += [f\"-j{self.parallel}\"]\r\n\r\n        build_temp = Path(self.build_temp) / ext.name\r\n        if not build_temp.exists():\r\n            build_temp.mkdir(parents=True)\r\n\r\n        subprocess.run(\r\n            [\"cmake\", ext.sourcedir] + cmake_args, cwd=build_temp, check=True\r\n        )\r\n        subprocess.run(\r\n            [\"cmake\", \"--build\", \".\"] + build_args, cwd=build_temp, check=True\r\n        )\r\n\r\n\r\nsetup(\r\n    # Currently, build_ext only provides an optional \"highest supported C++\r\n    # level\" feature, but in the future it may provide more features.\r\n    ext_modules=[CMakeExtension(\"mymodule\")],\r\n    cmdclass={\"build_ext\": CMakeBuild},\r\n    zip_safe=False,\r\n)\r\n```\r\n\r\n- `src/main.cpp`: The content is from `xtensor-python` [github](https://github.com/xtensor-stack/xtensor-python/blob/master/docs/source/examples/readme_example_1/main.cpp)\r\n```\r\n#include \u003cnumeric\u003e\r\n#include \u003cpybind11/pybind11.h\u003e\r\n#include \u003cxtensor/xmath.hpp\u003e\r\n#define FORCE_IMPORT_ARRAY\r\n#include \u003cxtensor-python/pyarray.hpp\u003e\r\n\r\ndouble sum_of_sines(xt::pyarray\u003cdouble\u003e \u0026m) {\r\n  auto sines = xt::sin(m); // sines does not actually hold values.\r\n  return std::accumulate(sines.begin(), sines.end(), 0.0);\r\n}\r\n\r\nPYBIND11_MODULE(mymodule, m) {\r\n  xt::import_numpy();\r\n  m.doc() = \"Test module for xtensor python bindings\";\r\n  m.def(\"sum_of_sines\", sum_of_sines, \"Sum the sines of the input values\");\r\n}\r\n```\r\n\r\n#### Installation:\r\n- Create venv, activate venv, and update pip:\r\n```shell\r\npython3 -m venv venv \u0026\u0026 source venv/bin/activate \u0026\u0026 python3 -m pip install --upgrade pip\r\n```\r\n- Install from Github repository directly:\r\n```\r\npython3 -m pip install \"mymodule @ git+ssh://git@github.com/lionlai1989/xtensor_pybind11_cmake.git\"\r\n```\r\n- (Optional) Git clone repository and pip install:\r\n```\r\ngit clone --recursive https://github.com/lionlai1989/xtensor_pybind11_cmake.git\r\npython3 -m pip install .\r\n```\r\n\r\n#### Verify installation:\r\n```python\r\n\u003e\u003e\u003e import numpy as np\r\n\u003e\u003e\u003e import mymodule\r\n\u003e\u003e\u003e v = np.arange(15).reshape(3, 5)\r\n\u003e\u003e\u003e mymodule.sum_of_sines(v)\r\n1.2853996391883833\r\n```\r\n***Everything works perfectly so far.***\r\n\r\n## Enable optimization and xsimd\r\nNow I want to enable optimization. Follow the [xtensor documentation](https://readthedocs.org/projects/xtensor/downloads/pdf/latest/)\r\n\r\n#### Add `xtensor::optimize` and `xtensor::use_xsimd`\r\n - Uncomment Line:44 in `CMakeLists.txt`:\r\n```\r\ntarget_link_libraries(mymodule PUBLIC xtensor pybind11::module xtensor-python Python::NumPy xtensor::optimize xtensor::use_xsimd)\r\n```\r\n\r\n - Run `pip install .` again, and the error shows `cannot find target xtensor::optimize`:\r\n```\r\nBuilding wheels for collected packages: mymodule\r\n  Building wheel for mymodule (pyproject.toml) ... error\r\n  error: subprocess-exited-with-error\r\n  \r\n  × Building wheel for mymodule (pyproject.toml) did not run successfully.\r\n  │ exit code: 1\r\n  ╰─\u003e [86 lines of output]\r\n      running bdist_wheel\r\n      running build\r\n      running build_ext\r\n      -- pybind11 v2.11.0 dev1\r\n      -- Found PythonInterp: /home/lai/xtensor_pybind11_cmake/venv/bin/python3 (found suitable version \"3.8.10\", minimum required is \"3.6\")\r\n      -- xtl v0.7.5\r\n      -- Building xtensor v0.24.6\r\n      -- Found xtl v0.7.5\r\n      -- xsimd v10.0.0\r\n      -- xtensor-python v0.26.1\r\n      -- Found xtensor v0.24.6\r\n      -- Found pybind11 v\r\n      -- Found PythonInterp: /home/lai/xtensor_pybind11_cmake/venv/bin/python3 (found version \"3.8.10\")\r\n      -- Found NumPy: version \"1.24.2\" /tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/numpy/core/include\r\n      -- Found numpy: /tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/numpy/core/include\r\n      -- Found Python: /home/lai/xtensor_pybind11_cmake/venv/bin/python3 (found version \"3.8.10\") found components: Interpreter Development NumPy Development.Module Development.Embed\r\n      -- Configuring done (0.4s)\r\n      CMake Error at CMakeLists.txt:43 (target_link_libraries):\r\n        Target \"mymodule\" links to:\r\n      \r\n          xtensor::optimize\r\n      \r\n        but the target was not found.  Possible reasons include:\r\n      \r\n          * There is a typo in the target name.\r\n          * A find_package call is missing for an IMPORTED target.\r\n          * An ALIAS target is missing.\r\n      \r\n      \r\n      \r\n      -- Generating done (0.0s)\r\n      CMake Generate step failed.  Build files cannot be regenerated correctly.\r\n      Traceback (most recent call last):\r\n        File \"/home/lai/xtensor_pybind11_cmake/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py\", line 353, in \u003cmodule\u003e\r\n          main()\r\n        File \"/home/lai/xtensor_pybind11_cmake/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py\", line 335, in main\r\n          json_out['return_val'] = hook(**hook_input['kwargs'])\r\n        File \"/home/lai/xtensor_pybind11_cmake/venv/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py\", line 251, in build_wheel\r\n          return _build_backend().build_wheel(wheel_directory, config_settings,\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/build_meta.py\", line 404, in build_wheel\r\n          return self._build_with_temp_dir(['bdist_wheel'], '.whl',\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/build_meta.py\", line 389, in _build_with_temp_dir\r\n          self.run_setup()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/build_meta.py\", line 334, in run_setup\r\n          exec(code, locals())\r\n        File \"\u003cstring\u003e\", line 127, in \u003cmodule\u003e\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/__init__.py\", line 87, in setup\r\n          return distutils.core.setup(**attrs)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py\", line 185, in setup\r\n          return run_commands(dist)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py\", line 201, in run_commands\r\n          dist.run_commands()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py\", line 973, in run_commands\r\n          self.run_command(cmd)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/dist.py\", line 1217, in run_command\r\n          super().run_command(command)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py\", line 992, in run_command\r\n          cmd_obj.run()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py\", line 343, in run\r\n          self.run_command(\"build\")\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py\", line 319, in run_command\r\n          self.distribution.run_command(command)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/dist.py\", line 1217, in run_command\r\n          super().run_command(command)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py\", line 992, in run_command\r\n          cmd_obj.run()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py\", line 132, in run\r\n          self.run_command(cmd_name)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py\", line 319, in run_command\r\n          self.distribution.run_command(command)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/dist.py\", line 1217, in run_command\r\n          super().run_command(command)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py\", line 992, in run_command\r\n          cmd_obj.run()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py\", line 84, in run\r\n          _build_ext.run(self)\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py\", line 346, in run\r\n          self.build_extensions()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py\", line 466, in build_extensions\r\n          self._build_extensions_serial()\r\n        File \"/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py\", line 492, in _build_extensions_serial\r\n          self.build_extension(ext)\r\n        File \"\u003cstring\u003e\", line 119, in build_extension\r\n        File \"/usr/lib/python3.8/subprocess.py\", line 516, in run\r\n          raise CalledProcessError(retcode, process.args,\r\n      subprocess.CalledProcessError: Command '['cmake', '/home/lai/xtensor_pybind11_cmake', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/lai/xtensor_pybind11_cmake/build/lib.linux-x86_64-cpython-38/', '-DPYTHON_EXECUTABLE=/home/lai/xtensor_pybind11_cmake/venv/bin/python3', '-DCMAKE_BUILD_TYPE=Release', '-DEXAMPLE_VERSION_INFO=0.0.1', '-GNinja', '-DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-d762ck7q/overlay/lib/python3.8/site-packages/ninja/data/bin/ninja']' returned non-zero exit status 1.\r\n      [end of output]\r\n  \r\n  note: This error originates from a subprocess, and is likely not a problem with pip.\r\n  ERROR: Failed building wheel for mymodule\r\nFailed to build mymodule\r\nERROR: Could not build wheels for mymodule, which is required to install pyproject.toml-based projects\r\n```\r\n\r\n#### Use `find_package` in `CMakeLists.txt`:\r\nMy previous experience tells me that using `find_package` could solve this problem. Thus, I tried:\r\n - Install `xtl`, `xtensor` and `xsimd` in the computer under`/tmp`:\r\n```\r\n(cd extern/xtl \u0026\u0026 cmake -D CMAKE_INSTALL_PREFIX=/tmp/xtl-install \u0026\u0026 make install)\r\n(cd extern/xtensor \u0026\u0026 cmake -D CMAKE_INSTALL_PREFIX=/tmp/xtensor-install -DCMAKE_PREFIX_PATH=/tmp/xtl-install \u0026\u0026 make install)\r\n(cd extern/xsimd \u0026\u0026 cmake -D CMAKE_INSTALL_PREFIX=/tmp/xsimd-install \u0026\u0026 make install)\r\n```\r\n - Delete `add_subdirectory` and add `find_package` in `CMakeLists.txt`:\r\n ```\r\nlist(APPEND CMAKE_PREFIX_PATH \"/tmp/xtl-install\")\r\nlist(APPEND CMAKE_PREFIX_PATH \"/tmp/xtensor-install\")\r\nlist(APPEND CMAKE_PREFIX_PATH \"/tmp/xsimd-install\")\r\nfind_package(xtl REQUIRED)\r\nfind_package(xtensor REQUIRED)\r\nfind_package(xsimd REQUIRED)\r\n```\r\n- Run `pip install .` again with `xtensor::optimize xtensor::use_xsimd`. And **everything works**. However, the problem is that `xtl`, `xtensor`, `xsimd` packages **cannot** be installed before users running `pip install` command. \r\n\r\n#### Try FetchContent\r\nI wan thinking if `add_subdirectory` doesn't work, maybe `FetchContent` will work. Unfortunately, it's not the case. The same error message still exists. You can uncomment the code block of `FetchContent` in `CMakeLists.txt` to test it out.\r\n\r\n### My questions are:  \r\n 1. How can I make the target `mymodule` link to `xtensor::optimize` and  `xtensor::use_xsimd` when using `add_subdirectory` or `FetchContent`?  \r\n 2. If 1.) is not possible, is there a way to automate the installation of `xtensor`, `xtl`, and `xsimd` in CMake and pip installation process so that `find_package` can be used in `CMakeLists.txt`. Subsequently, anyone can install this package with a single and simple `pip install`.\r\n\r\nSystem Information:\r\n - Ubuntu 20.04\r\n - Python 3.8.10\r\n - All submodules are at master branch.\r\n - All other system build requirements are fixed by `pyproject.toml`.\r\n\r\n(I also posted the same question on [StackOverflow](https://stackoverflow.com/questions/75955406/build-c-xtensor-project-as-python-package-so-users-can-pip-install-without-a))\r\n ","author":{"url":"https://github.com/lionlai1989","@type":"Person","name":"lionlai1989"},"datePublished":"2023-04-07T04:12:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/293/xtensor-python/issues/293"}

route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:ecb00b67-1e78-9c88-6a7b-a757aa48b45b
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-id9A42:E8D97:52272C:72883D:696F3375
html-safe-nonce1523d57087c24cdba7c222dbb60846dd04e9abe24c2d0045ef19806e4f5fb040
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5QTQyOkU4RDk3OjUyMjcyQzo3Mjg4M0Q6Njk2RjMzNzUiLCJ2aXNpdG9yX2lkIjoiNTEyNTk2MzcyNDg4NzExMDUxNyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac113fc6087c6a57399cf31747b13226f75a9e46ac2583511fe95be8b9c05d4355
hovercard-subject-tagissue:1658337752
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/xtensor-stack/xtensor-python/293/issue_layout
twitter:imagehttps://opengraph.githubassets.com/0ae766bdf379da658833bcf377497ad8b0c0498e4aa88b09942cf18cdc266fe5/xtensor-stack/xtensor-python/issues/293
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/0ae766bdf379da658833bcf377497ad8b0c0498e4aa88b09942cf18cdc266fe5/xtensor-stack/xtensor-python/issues/293
og:image:altI am following XTensor-python tutorial to build a XTensor C++ project which has Python binding with CMake. The end goal is to create a self-contained Python package that users can install via pip i...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamelionlai1989
hostnamegithub.com
expected-hostnamegithub.com
Noneb278ad162d35332b6de714dfb005de04386c4d92df6475522bef910f491a35ee
turbo-cache-controlno-preview
go-importgithub.com/xtensor-stack/xtensor-python git https://github.com/xtensor-stack/xtensor-python.git
octolytics-dimension-user_id56542697
octolytics-dimension-user_loginxtensor-stack
octolytics-dimension-repository_id73431892
octolytics-dimension-repository_nwoxtensor-stack/xtensor-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id73431892
octolytics-dimension-repository_network_root_nwoxtensor-stack/xtensor-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release39aed5006635ab6f45e6b77d23e73b08a00272a3
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293#start-of-content
https://patch-diff.githubusercontent.com/
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fxtensor-stack%2Fxtensor-python%2Fissues%2F293
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fxtensor-stack%2Fxtensor-python%2Fissues%2F293
Sign up https://patch-diff.githubusercontent.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=xtensor-stack%2Fxtensor-python
Reloadhttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293
Reloadhttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293
Reloadhttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293
xtensor-stack https://patch-diff.githubusercontent.com/xtensor-stack
xtensor-pythonhttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python
Notifications https://patch-diff.githubusercontent.com/login?return_to=%2Fxtensor-stack%2Fxtensor-python
Fork 67 https://patch-diff.githubusercontent.com/login?return_to=%2Fxtensor-stack%2Fxtensor-python
Star 364 https://patch-diff.githubusercontent.com/login?return_to=%2Fxtensor-stack%2Fxtensor-python
Code https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python
Issues 38 https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues
Pull requests 6 https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/pulls
Actions https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/actions
Projects 0 https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/projects
Security Uh oh! There was an error while loading. Please reload this page. https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/security
Please reload this pagehttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293
Insights https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/pulse
Code https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python
Issues https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues
Pull requests https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/pulls
Actions https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/actions
Projects https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/projects
Security https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/security
Insights https://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/pulse
New issuehttps://patch-diff.githubusercontent.com/login?return_to=https://github.com/xtensor-stack/xtensor-python/issues/293
New issuehttps://patch-diff.githubusercontent.com/login?return_to=https://github.com/xtensor-stack/xtensor-python/issues/293
Build C++ XTensor project as Python package so users can pip install without any additional dependencieshttps://patch-diff.githubusercontent.com/xtensor-stack/xtensor-python/issues/293#top
https://github.com/lionlai1989
https://github.com/lionlai1989
lionlai1989https://github.com/lionlai1989
on Apr 7, 2023https://github.com/xtensor-stack/xtensor-python/issues/293#issue-1658337752
XTensor-pythonhttps://github.com/xtensor-stack/xtensor-python
tutorialhttps://xtensor-python.readthedocs.io/en/latest/
You can follow the instruction herehttps://github.com/lionlai1989/xtensor_pybind11_cmake
xtensor-python tutorialhttps://github.com/xtensor-stack/xtensor-python/blob/master/docs/source/examples/readme_example_1/CMakeLists.txt
githubhttps://github.com/pybind/cmake_example/blob/master/setup.py
githubhttps://github.com/xtensor-stack/xtensor-python/blob/master/docs/source/examples/readme_example_1/main.cpp
xtensor documentationhttps://readthedocs.org/projects/xtensor/downloads/pdf/latest/
StackOverflowhttps://stackoverflow.com/questions/75955406/build-c-xtensor-project-as-python-package-so-users-can-pip-install-without-a
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.