493 lines
18 KiB
ReStructuredText
493 lines
18 KiB
ReStructuredText
|
|
Recipes
|
|
=======
|
|
|
|
This page describes how python-for-android (p4a) compilation recipes
|
|
work, and how to build your own. If you just want to build an APK,
|
|
ignore this and jump straight to the :doc:`quickstart`.
|
|
|
|
Recipes are special scripts for compiling and installing different programs
|
|
(including Python modules) into a p4a distribution. They are necessary
|
|
to take care of compilation for any compiled components, as these must
|
|
be compiled for Android with the correct architecture.
|
|
|
|
python-for-android comes with many recipes for popular modules. No
|
|
recipe is necessary to use of Python modules with no
|
|
compiled components; these are installed automaticaly via pip.
|
|
|
|
If you are new to building recipes, it is recommended that you first
|
|
read all of this page, at least up to the Recipe reference
|
|
documentation. The different recipe sections include a number of
|
|
examples of how recipes are built or overridden for specific purposes.
|
|
|
|
|
|
Creating your own Recipe
|
|
------------------------
|
|
|
|
The formal reference documentation of the Recipe
|
|
class can be found in the `Recipe class <recipe_class_>`_ section and below.
|
|
|
|
Check the `recipe template section <recipe_template_>`_ for a template
|
|
that combines all of these ideas, in which you can replace whichever
|
|
components you like.
|
|
|
|
The basic declaration of a recipe is as follows::
|
|
|
|
class YourRecipe(Recipe):
|
|
|
|
url = 'http://example.com/example-{version}.tar.gz'
|
|
version = '2.0.3'
|
|
md5sum = '4f3dc9a9d857734a488bcbefd9cd64ed'
|
|
|
|
patches = ['some_fix.patch'] # Paths relative to the recipe dir
|
|
|
|
depends = ['kivy', 'sdl2'] # These are just examples
|
|
conflicts = ['pygame']
|
|
|
|
recipe = YourRecipe()
|
|
|
|
See the `Recipe class documentation <recipe_class_>`_ for full
|
|
information about each parameter.
|
|
|
|
These core options are vital for all recipes, though the url may be
|
|
omitted if the source is somehow loaded from elsewhere.
|
|
|
|
You must include ``recipe = YourRecipe()``. This variable is accessed
|
|
when the recipe is imported.
|
|
|
|
.. note:: The url includes the ``{version}`` tag. You should only
|
|
access the url with the ``versioned_url`` property, which
|
|
replaces this with the version attribute.
|
|
|
|
The actual build process takes place via three core methods::
|
|
|
|
def prebuild_arch(self, arch):
|
|
super(YourRecipe, self).prebuild_arch(arch)
|
|
# Do any pre-initialisation
|
|
|
|
def build_arch(self, arch):
|
|
super(YourRecipe, self).build_arch(arch)
|
|
# Do the main recipe build
|
|
|
|
def postbuild_arch(self, arch):
|
|
super(YourRecipe, self).build_arch(arch)
|
|
# Do any clearing up
|
|
|
|
These methods are always run in the listed order; prebuild, then
|
|
build, then postbuild.
|
|
|
|
If you defined an url for your recipe, you do *not* need to manually
|
|
download it, this is handled automatically.
|
|
|
|
The recipe will automatically be built in a special isolated build
|
|
directory, which you can access with
|
|
:code:`self.get_build_dir(arch.arch)`. You should only work within
|
|
this directory. It may be convenient to use the ``current_directory``
|
|
context manager defined in toolchain.py::
|
|
|
|
from pythonforandroid.toolchain import current_directory
|
|
def build_arch(self, arch):
|
|
super(YourRecipe, self).build_arch(arch)
|
|
with current_directory(self.get_build_dir(arch.arch)):
|
|
with open('example_file.txt', 'w'):
|
|
fileh.write('This is written to a file within the build dir')
|
|
|
|
The argument to each method, ``arch``, is an object relating to the
|
|
architecture currently being built for. You can mostly ignore it,
|
|
though may need to use the arch name ``arch.arch``.
|
|
|
|
.. note:: You can also implement arch-specific versions of each
|
|
method, which are called (if they exist) by the superclass,
|
|
e.g. ``def prebuild_armeabi(self, arch)``.
|
|
|
|
|
|
This is the core of what's necessary to write a recipe, but has not
|
|
covered any of the details of how one actually writes code to compile
|
|
for android. This is covered in the next sections, including the
|
|
`standard mechanisms <standard_mechanisms_>`_ used as part of the
|
|
build, and the details of specific recipe classes for Python, Cython,
|
|
and some generic compiled recipes. If your module is one of the
|
|
latter, you should use these later classes rather than reimplementing
|
|
the functionality from scratch.
|
|
|
|
.. _standard_mechanisms:
|
|
|
|
Methods and tools to help with compilation
|
|
------------------------------------------
|
|
|
|
Patching modules before installation
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
You can easily apply patches to your recipes by adding them to the
|
|
``patches`` declaration, e.g.::
|
|
|
|
patches = ['some_fix.patch',
|
|
'another_fix.patch']
|
|
|
|
The paths should be relative to the recipe file. Patches are
|
|
automatically applied just once (i.e. not reapplied the second time
|
|
python-for-android is run).
|
|
|
|
You can also use the helper functions in ``pythonforandroid.patching``
|
|
to apply patches depending on certain conditions, e.g.::
|
|
|
|
from pythonforandroid.patching import will_build, is_arch
|
|
|
|
...
|
|
|
|
class YourRecipe(Recipe):
|
|
|
|
patches = [('x86_patch.patch', is_arch('x86')),
|
|
('sdl2_compatibility.patch', will_build('sdl2'))]
|
|
|
|
...
|
|
|
|
You can include your own conditions by passing any function as the
|
|
second entry of the tuple. It will receive the ``arch`` (e.g. x86,
|
|
armeabi) and ``recipe`` (i.e. the Recipe object) as kwargs. The patch
|
|
will be applied only if the function returns True.
|
|
|
|
|
|
Installing libs
|
|
~~~~~~~~~~~~~~~
|
|
|
|
Some recipes generate .so files that must be manually copied into the
|
|
android project. You can use code like the following to accomplish
|
|
this, copying to the correct lib cache dir::
|
|
|
|
def build_arch(self, arch):
|
|
do_the_build() # e.g. running ./configure and make
|
|
|
|
import shutil
|
|
shutil.copyfile('a_generated_binary.so',
|
|
self.ctx.get_libs_dir(arch.arch))
|
|
|
|
Any libs copied to this dir will automatically be included in the
|
|
appropriate libs dir of the generated android project.
|
|
|
|
Compiling for the Android architecture
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
When performing any compilation, it is vital to do so with appropriate
|
|
environment variables set, ensuring that the Android libraries are
|
|
properly linked and the compilation target is the correct
|
|
architecture.
|
|
|
|
You can get a dictionary of appropriate environment variables with the
|
|
``get_recipe_env`` method. You should make sure to set this
|
|
environment for any processes that you call. It is convenient to do
|
|
this using the ``sh`` module as follows::
|
|
|
|
def build_arch(self, arch):
|
|
super(YourRecipe, self).build_arch(arch)
|
|
env = self.get_recipe_env(arch)
|
|
sh.echo('$PATH', _env=env) # Will print the PATH entry from the
|
|
# env dict
|
|
|
|
You can also use the ``shprint`` helper function from the p4a
|
|
toolchain module, which will print information about the process and
|
|
its current status::
|
|
|
|
from pythonforandroid.toolchain import shprint
|
|
shprint(sh.echo, '$PATH', _env=env)
|
|
|
|
You can also override the ``get_recipe_env`` method to add new env
|
|
vars for the use of your recipe. For instance, the Kivy recipe does
|
|
the following when compiling for SDL2, in order to tell Kivy what
|
|
backend to use::
|
|
|
|
def get_recipe_env(self, arch):
|
|
env = super(KivySDL2Recipe, self).get_recipe_env(arch)
|
|
env['USE_SDL2'] = '1'
|
|
|
|
env['KIVY_SDL2_PATH'] = ':'.join([
|
|
join(self.ctx.bootstrap.build_dir, 'jni', 'SDL', 'include'),
|
|
join(self.ctx.bootstrap.build_dir, 'jni', 'SDL2_image'),
|
|
join(self.ctx.bootstrap.build_dir, 'jni', 'SDL2_mixer'),
|
|
join(self.ctx.bootstrap.build_dir, 'jni', 'SDL2_ttf'),
|
|
])
|
|
return env
|
|
|
|
.. warning:: When using the sh module like this the new env *completely
|
|
replaces* the normal environment, so you must define any env
|
|
vars you want to access.
|
|
|
|
Including files with your recipe
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The should_build method
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The Recipe class has a ``should_build`` method, which returns a
|
|
boolean. This is called for each architecture before running
|
|
``build_arch``, and if it returns False then the build is
|
|
skipped. This is useful to avoid building a recipe more than once for
|
|
different dists.
|
|
|
|
By default, should_build returns True, but you can override it however
|
|
you like. For instance, PythonRecipe and its subclasses all replace it
|
|
with a check for whether the recipe is already installed in the Python
|
|
distribution::
|
|
|
|
def should_build(self, arch):
|
|
name = self.site_packages_name
|
|
if name is None:
|
|
name = self.name
|
|
if self.ctx.has_package(name):
|
|
info('Python package already exists in site-packages')
|
|
return False
|
|
info('{} apparently isn\'t already in site-packages'.format(name))
|
|
return True
|
|
|
|
|
|
|
|
Using a PythonRecipe
|
|
--------------------
|
|
|
|
If your recipe is to install a Python module without compiled
|
|
components, you should use a PythonRecipe. This overrides
|
|
``build_arch`` to automatically call the normal ``python setup.py
|
|
install`` with an appropriate environment.
|
|
|
|
For instance, the following is all that's necessary to create a recipe
|
|
for the Vispy module::
|
|
|
|
from pythonforandroid.toolchain import PythonRecipe
|
|
class VispyRecipe(PythonRecipe):
|
|
version = 'master'
|
|
url = 'https://github.com/vispy/vispy/archive/{version}.zip'
|
|
|
|
depends = ['python2', 'numpy']
|
|
|
|
site_packages_name = 'vispy'
|
|
|
|
recipe = VispyRecipe()
|
|
|
|
The ``site_packages_name`` is a new attribute that identifies the
|
|
folder in which the module will be installed in the Python
|
|
package. This is only essential to add if the name is different to the
|
|
recipe name. It is used to check if the recipe installation can be
|
|
skipped, which is the case if the folder is already present in the
|
|
Python installation.
|
|
|
|
For reference, the code that accomplishes this is the following::
|
|
|
|
def build_arch(self, arch):
|
|
super(PythonRecipe, self).build_arch(arch)
|
|
self.install_python_package()
|
|
|
|
def install_python_package(self):
|
|
'''Automate the installation of a Python package (or a cython
|
|
package where the cython components are pre-built).'''
|
|
arch = self.filtered_archs[0]
|
|
env = self.get_recipe_env(arch)
|
|
|
|
info('Installing {} into site-packages'.format(self.name))
|
|
|
|
with current_directory(self.get_build_dir(arch.arch)):
|
|
hostpython = sh.Command(self.ctx.hostpython)
|
|
|
|
shprint(hostpython, 'setup.py', 'install', '-O2', _env=env)
|
|
|
|
This combines techniques and tools from the above documentation to
|
|
create a generic mechanism for all Python modules.
|
|
|
|
.. note:: The hostpython is the path to the Python binary that should
|
|
be used for any kind of installation. You *must* run Python
|
|
in a similar way if you need to do so in any of your own
|
|
recipes.
|
|
|
|
|
|
Using a CythonRecipe
|
|
--------------------
|
|
|
|
If your recipe is to install a Python module that uses Cython, you
|
|
should use a CythonRecipe. This overrides ``build_arch`` to both build
|
|
the cython components and to install the Python module just like a
|
|
normal PythonRecipe.
|
|
|
|
For instance, the following is all that's necessary to make a recipe
|
|
for Kivy (in this case, depending on Pygame rather than SDL2)::
|
|
|
|
class KivyRecipe(CythonRecipe):
|
|
version = 'stable'
|
|
url = 'https://github.com/kivy/kivy/archive/{version}.zip'
|
|
name = 'kivy'
|
|
|
|
depends = ['pygame', 'pyjnius', 'android']
|
|
|
|
recipe = KivyRecipe()
|
|
|
|
For reference, the code that accomplishes this is the following::
|
|
|
|
def build_arch(self, arch):
|
|
Recipe.build_arch(self, arch) # a hack to avoid calling
|
|
# PythonRecipe.build_arch
|
|
self.build_cython_components(arch)
|
|
self.install_python_package() # this is the same as in a PythonRecipe
|
|
|
|
def build_cython_components(self, arch):
|
|
env = self.get_recipe_env(arch)
|
|
with current_directory(self.get_build_dir(arch.arch)):
|
|
hostpython = sh.Command(self.ctx.hostpython)
|
|
|
|
# This first attempt *will* fail, because cython isn't
|
|
# installed in the hostpython
|
|
try:
|
|
shprint(hostpython, 'setup.py', 'build_ext', _env=env)
|
|
except sh.ErrorReturnCode_1:
|
|
pass
|
|
|
|
# ...so we manually run cython from the user's system
|
|
shprint(sh.find, self.get_build_dir('armeabi'), '-iname', '*.pyx', '-exec',
|
|
self.ctx.cython, '{}', ';', _env=env)
|
|
|
|
# now cython has already been run so the build works
|
|
shprint(hostpython, 'setup.py', 'build_ext', '-v', _env=env)
|
|
|
|
# stripping debug symbols lowers the file size a lot
|
|
build_lib = glob.glob('./build/lib*')
|
|
shprint(sh.find, build_lib[0], '-name', '*.o', '-exec',
|
|
env['STRIP'], '{}', ';', _env=env)
|
|
|
|
The failing build and manual cythonisation is necessary, first to
|
|
make sure that any .pyx files have been generated by setup.py, and
|
|
second because cython isn't installed in the hostpython build.
|
|
|
|
This may actually fail if the setup.py tries to import cython before
|
|
making any pyx files (in which case it crashes too early), although
|
|
this is probably not usually an issue. If this happens to you, try
|
|
patching to remove this import or make it fail quietly.
|
|
|
|
Other than this, these methods follow the techniques in the above
|
|
documentation to make a generic recipe for most cython based modules.
|
|
|
|
Using a CompiledComponentsPythonRecipe
|
|
--------------------------------------
|
|
|
|
This is similar to a CythonRecipe but is intended for modules like
|
|
numpy which include compiled but non-cython components. It uses a
|
|
similar mechanism to compile with the right environment.
|
|
|
|
This isn't documented yet because it will probably be changed so that
|
|
CythonRecipe inherits from it (to avoid code duplication).
|
|
|
|
|
|
Using an NDKRecipe
|
|
------------------
|
|
|
|
If you are writing a recipe not for a Python module but for something
|
|
that would normall go in the JNI dir of an Android project (i.e. it
|
|
has an ``Application.mk`` and ``Android.mk`` that the Android build
|
|
system can use), you can use an NDKRecipe to automatically set it
|
|
up. The NDKRecipe overrides the normal ``get_build_dir`` method to
|
|
place things in the Android project.
|
|
|
|
.. warning:: The NDKRecipe does *not* currently actually call
|
|
ndk-build, you must add this call (for your module) by
|
|
manually making a build_arch method. This may be fixed
|
|
later.
|
|
|
|
For instance, the following recipe is all that's necessary to place
|
|
SDL2_ttf in the jni dir. This is built later by the SDL2 recipe, which
|
|
calls ndk-build with this as a dependency::
|
|
|
|
class LibSDL2TTF(NDKRecipe):
|
|
version = '2.0.12'
|
|
url = 'https://www.libsdl.org/projects/SDL_ttf/release/SDL2_ttf-{version}.tar.gz'
|
|
dir_name = 'SDL2_ttf'
|
|
|
|
recipe = LibSDL2TTF()
|
|
|
|
The dir_name argument is a new class attribute that tells the recipe
|
|
what the jni dir folder name should be. If it is omitted, the recipe
|
|
name is used. Be careful here, sometimes the folder name is important,
|
|
especially if this folder is a dependency of something else.
|
|
|
|
.. _recipe_template:
|
|
|
|
A Recipe template
|
|
-----------------
|
|
|
|
The following template includes all the recipe sections you might
|
|
use. None are compulsory, feel free to delete method
|
|
overrides if you do not use them::
|
|
|
|
from pythonforandroid.toolchain import Recipe, shprint, current_directory
|
|
from os.path import exists, join
|
|
import sh
|
|
import glob
|
|
|
|
|
|
class YourRecipe(Recipe):
|
|
# This could also inherit from PythonRecipe etc. if you want to
|
|
# use their pre-written build processes
|
|
|
|
version = 'some_version_string'
|
|
url = 'http://example.com/example-{version}.tar.gz'
|
|
# {version} will be replaced with self.version when downloading
|
|
|
|
depends = ['python2', 'numpy'] # A list of any other recipe names
|
|
# that must be built before this
|
|
# one
|
|
|
|
conflicts = [] # A list of any recipe names that cannot be built
|
|
# alongside this one
|
|
|
|
def get_recipe_env(self, arch):
|
|
env = super(YourRecipe, self).get_recipe_env()
|
|
# Manipulate the env here if you want
|
|
return env
|
|
|
|
def should_build(self):
|
|
# Add a check for whether the recipe is already built if you
|
|
# want, and return False if it is.
|
|
return True
|
|
|
|
def prebuild_arch(self, arch):
|
|
super(YourRecipe, self).prebuild_arch(self)
|
|
# Do any extra prebuilding you want, e.g.:
|
|
self.apply_patch('path/to/patch.patch')
|
|
|
|
def build_arch(self, arch):
|
|
super(YourRecipe, self).build_arch(self)
|
|
# Build the code. Make sure to use the right build dir, e.g.
|
|
with current_directory(self.get_build_dir(arch.arch)):
|
|
sh.ls('-lathr') # Or run some commands that actually do
|
|
# something
|
|
|
|
def postbuild_arch(self, arch):
|
|
super(YourRecipe, self).prebuild_arch(self)
|
|
# Do anything you want after the build, e.g. deleting
|
|
# unnecessary files such as documentation
|
|
|
|
|
|
recipe = YourRecipe()
|
|
|
|
|
|
Examples of recipes
|
|
-------------------
|
|
|
|
This documentation covers most of what is ever necessary to make a
|
|
recipe work. For further examples, python-for-android includes many
|
|
recipes for popular modules, which are an excellent resource to find
|
|
out how to add your own. You can find these in the `python-for-android
|
|
Github page
|
|
<https://github.com/kivy/python-for-android/tree/master/pythonforandroid/recipes>`__.
|
|
|
|
|
|
.. _recipe_class:
|
|
|
|
The ``Recipe`` class
|
|
--------------------
|
|
|
|
The ``Recipe`` is the base class for all p4a recipes. The core
|
|
documentation of this class is given below, followed by discussion of
|
|
how to create your own Recipe subclass.
|
|
|
|
.. autoclass:: toolchain.Recipe
|
|
:members:
|
|
:member-order: = 'bysource'
|
|
|
|
|
|
|