5. Reference¶
5.1. Polyversion (library)¶
polyversion |
Python-2.7-safe, no-deps code to discover sub-project versions in Git polyvers monorepos. |
polyversion.polyversion (**kw) |
Report the pvtag of the pname in the git repo hosting the source-file calling this. |
polyversion.polytime (**kw) |
The timestamp of last commit in git repo hosting the source-file calling this. |
polyversion.pkg_metadata_version (pname[, …]) |
Get the version from package metadata if present. |
polyversion.setuplugin |
A setuptools plugin with x2 setup() kwds and monkeypatch all bdist... cmds. |
polyversion.setuplugin.init_plugin_kw (dist, …) |
A setuptools kwd for deriving subproject versions from PKG-INFO or git tags. |
polyversion.setuplugin.check_bdist_kw (dist, …) |
A setuptools kwd for aborting bdist… commands if not on r-tag. |
5.1.1. Module: polyversion
¶
Python-2.7-safe, no-deps code to discover sub-project versions in Git polyvers monorepos.
The polyvers version-configuration tool is generating pvtags like:
proj-foo-v0.1.0
And assuming polyversion()
is invoked from within a Git repo, it may return
either 0.1.0
or 0.1.0+2.gcaffe00
, if 2 commits have passed since
last pvtag.
Also, this library function as a setuptools “plugin” (see setuplugin
).
Finally, the wheel can be executed like that:
python polyversion-*.whl --help
-
polyversion.
polyversion
(**kw)[source]¶ Report the pvtag of the pname in the git repo hosting the source-file calling this.
Parameters: - pname (str) –
The project-name, used as the prefix of pvtags when searching them. If not given, defaults to the last segment of the module-name of the caller.
Attention
when calling it from
setup.py
files, auto-deduction above will not work; you must supply a project name. - default_version (str) –
What version to return if git cmd fails. Set it to None to raise if no vtag found.
Tip
For cases where a shallow git-clone does not finds any vtags back in history, or simply because the project is new, and there are no vtags, we set default-version to empty-string, to facilitate pip-installing these projects from sources.
- default_version_env_var (str) – Override which env-var to read version from, if git cmd fails
[Default:
<pname>_VERSION
] - mono_project (bool) –
- false: (default) monorepo, ie multiple sub-projects per git-repo.
Tags formatted by pvtags
pvtag_format
&pvtag_regex
(likepname-v1.2.3
). - true: mono-project, ie only one project in git-repo
Tags formatted as vtags
vtag_format
&vtag_regex
. (likev1.2.3
).
- false: (default) monorepo, ie multiple sub-projects per git-repo.
Tags formatted by pvtags
- tag_format (str) –
The PEP 3101 pattern for creating pvtags (or vtags).
- It receives 3 parameters to interpolate:
{pname}, {vprefix}, {version} = '*'
. - It is used also to generate the match patterns for
git describe --match <pattern>
command. - It overrides mono_project arg.
- See
pvtag_format
&vtag_format
- It receives 3 parameters to interpolate:
- tag_regex (regex) –
The regex pattern breaking apart pvtags, with 3 named capturing groups:
pname
,version
(without the ‘v’),descid
(optional) anything following the dash(‘-‘) after the version ingit-describe
result.- It is given 2 PEP 3101 parameters
{pname}, {vprefix}
to interpolate. - It overrides mono_project arg.
- See PEP 0426 for project-name characters and format.
- See
pvtag_regex
&vtag_regex
- vprefixes (str) – a 2-element array of str -
tag_vprefixes
assumed when not specified - is_release –
a 3-state boolean used as index into
tag_vprefixes
:- false: v-tags searched;
- true: r-tags searched;
- None: both tags searched.
- basepath (str) – The path of the outermost package inside the git repo hosting the project; if missing, assumed as the dirname of the calling code’s package.
- git_options – a str or an iterator of (converted to str) options to pass
to
git describe
command (empty by default). If a string, it is splitted by spaces. - return_all – when true, return the 3-tuple (tag, version, desc-id) (not just version)
Returns: The version-id (or 3-tuple) derived from the pvtag, or default if command failed/returned nothing, unless None, in which case, it raises.
Raises: CalledProcessError – if it cannot find any vtag and default_version is None (e.g. no git cmd/repo, no valid tags)
Tip
It is to be used, for example, in package
__init__.py
files like this:__version__ = polyversion()
Or from any other file:
__version__ = polyversion('myproj')
Note
This is a python==2.7 & python<3.6 safe function; there is also the similar function with elaborate error-handling
polyvers.pvtags.describe_project()
in the full-blown tool polyvers.- pname (str) –
-
polyversion.
polytime
(**kw)[source]¶ The timestamp of last commit in git repo hosting the source-file calling this.
Parameters: - no_raise (str) – If true, never fail and return current-time. Assumed true if a default version env-var is found.
- basepath (str) – The path of the outermost package inside the git repo hosting the project; if missing, assumed as the dirname of the calling code’s package.
- pname (str) –
The project-name used only as the prefix for default version env-var. If not given, defaults to the last segment of the module-name of the caller. Another alternative is to use directly the default_version_env_var kwd.
Attention
when calling it from
setup.py
files, auto-deduction above will not work; you must supply a project name. - default_version_env_var (str) – Override which env-var to read version from, if git cmd fails
[Default:
<pname>_VERSION
]
Returns: the commit-date if in git repo, or now; RFC 2822 formatted
-
polyversion.
decide_vprefixes
(vprefixes, is_release)[source]¶ Decide v-tag, r-tag or both; no surprises params, return always an array.
-
polyversion.
vtag_format
= '{vprefix}{version}'¶ Like
pvtag_format
but for mono-project version-tags.
-
polyversion.
vtag_regex
= '(?xmi)\n ^(?P<pname>)\n {vprefix}(?P<version>\\d[^-]*)\n (?:-(?P<descid>\\d+-g[a-f\\d]+))?$\n'¶ Like
pvtag_format
but for mono-project version-tags.
5.1.2. Module: polyversion.setuplugin
¶
A setuptools plugin with x2 setup()
kwds and monkeypatch all bdist...
cmds.
Tip
Set envvar[DISTUTILS_DEBUG] to debug it. From https://docs.python.org/3.7/distutils/setupscript.html#debugging-the-setup-script
-
polyversion.setuplugin.
init_plugin_kw
(dist, attr, kw_value)[source]¶ A setuptools kwd for deriving subproject versions from PKG-INFO or git tags.
Parameters: - dist – class:distutils.Distribution
- attr (str) – the name of the keyword
- kw_value –
The content of the new
setup(polyversion=...)
keyword.SYNTAX:
'polyversion': (<bool> | <dict>)
When it is a dict, its keys roughly mimic those in
polyversion()
except those differences:param pname: absent; derived from setup(name=...)
keywordparam default_version: absent; derived from setup(version=...)
keyword:- if None/not given, any problems will be raised,
and
setup.py
script wil abort - if
version
is a (possibly empty) string, this will be used in case version cannot be auto-retrieved.
param is_release: absent; always False when deriving the version, and True when bdist-checking param basepath: if not given, derived from setup(package_dirs={...})
keyword or ‘.’ (and never from caller-stack).See
polyversion()
for keyword-dict’s content. - if None/not given, any problems will be raised,
and
It tries first to see if project contained in a distribution-archive (e.g. a “wheel”), and tries to derive the version from egg-infos. Then it falls through retrieving it from git tags.
Tip
For cases where a shallow git-clone does not finds any vtags back in history, or simply because the project is new, and there are no vtags, we set default-version to empty-string, to facilitate pip-installing these projects from sources.
-
polyversion.setuplugin.
check_bdist_kw
(dist, _attr, kw_value)[source]¶ A setuptools kwd for aborting bdist… commands if not on r-tag.
SYNTAX:
'polyversion_check_bdist_enabled': <any>
When <any> evaluates to false (default), any bdist… (e.g.
bdist_wheel
), setuptools commands will abort if not run from a release tag.By default it this check is bypassed. To enable it, without editing your sources add this in your
$CWD/setup.cfg
file:[global] polyversion_check_bdist_enabled = true ...
- Ignored, if polyversion kw is not enabled.
- Registered in distutils.setup_keywords entry_point of this project’s
setup.py
file.
5.2. Polyvers (command-tool)¶
polyvers |
Top-level package for polyvers version-configuration tool. |
polyvers.cli |
The code of polyvers shell-commands. |
polyvers.bumpcmd |
The command that actually bumps versions. |
polyvers.pvproject |
The main structures of polyvers. |
polyvers.pvtags |
Git code to make/inspect sub-project “(p)vtags” and respective commits in (mono)repos. |
polyvers.engrave |
Search and replace version-ids in files. |
polyvers.vermath |
Validate absolute versions or add relative ones on top of a base absolute. |
polyvers.cmdlet.cfgcmd |
Commands to inspect configurations and other cli infos. |
polyvers.cmdlet.cmdlets |
Utils for building elaborate Commands/Sub-commands with traitlets [1] Application. |
polyvers.cmdlet.errlog |
Suppress or ignore exceptions collected in a nested contexts. |
polyvers.cmdlet.interpctxt |
Enable Unicode-trait to pep3101-interpolate {key} patterns from “context” dicts. |
polyvers.utils.mainpump |
Utils pumping results out of yielding functions for main(). |
polyvers.utils.logconfutils |
Utils for configuring and using elaborate logs and handling main() failures. |
polyvers.utils.oscmd |
Utility to call OS commands through subprocess.run() with logging. |
polyvers.utils.fileutil |
Generic utils. |
5.2.2. Module: polyvers.cli
¶
The code of polyvers shell-commands.
-
class
polyvers.cli.
InitCmd
(*args, **kw)[source]¶ Generate configurations based on directory contents.
-
initialize
(argv=None)[source]¶ Invoked after __init__() by make_cmd() to apply configs and build subapps.
Parameters: argv – If undefined, they are replaced with sys.argv[1:]
!It parses cl-args before file-configs, to detect sub-commands and update any
config_paths
, then it reads all file-configs, and then re-apply cmd-line configs as overrides (trick copied from jupyter-core).
-
-
class
polyvers.cli.
LogconfCmd
(*args, **kw)[source]¶ Write a logging-configuration file that can filter logs selectively.
-
class
polyvers.cli.
PolyversCmd
(**kwargs)[source]¶ Bump independently PEP-440 versions of sub-project in Git monorepos.
- SYNTAX:
- {cmd_chain} <sub-cmd> …
-
bootstrapp_projects
() → None[source]¶ Ensure valid configuration exist for monorepo/mono-project(s).
Raises: CmdException – if cwd not inside a git repo
-
class
polyvers.cli.
StatusCmd
(*args, **kw)[source]¶ List the versions of project(s).
- SYNTAX:
- {cmd_chain} [OPTIONS] [<project>]…
-
polyvers.cli.
merge_dict
(dct, merge_dct)[source]¶ Recursive dict merge. Inspired by :meth:
dict.update()
, instead of updating only top-level keys, dict_merge recurses down into dicts nested to an arbitrary depth, updating keys. Themerge_dct
is merged intodct
. :param dct: dict onto which the merge is executed :param merge_dct: dct merged into dct :return: NoneAdapted from: https://gist.github.com/angstwad/bf22d1822c38a92ec0a9
5.2.2. Module: polyvers.bumpcmd
¶
The code of polyvers shell-commands.
-
class
polyvers.cli.
InitCmd
(*args, **kw)[source] Generate configurations based on directory contents.
-
initialize
(argv=None)[source] Invoked after __init__() by make_cmd() to apply configs and build subapps.
Parameters: argv – If undefined, they are replaced with sys.argv[1:]
!It parses cl-args before file-configs, to detect sub-commands and update any
config_paths
, then it reads all file-configs, and then re-apply cmd-line configs as overrides (trick copied from jupyter-core).
-
run
(*args)[source] Leaf sub-commands must inherit this instead of
start()
without invokingsuper()
.Parameters: args – Invoked by start()
withextra_args
.By default, screams about using sub-cmds, or about doing nothing!
-
-
class
polyvers.cli.
LogconfCmd
(*args, **kw)[source] Write a logging-configuration file that can filter logs selectively.
-
run
(*args)[source] Leaf sub-commands must inherit this instead of
start()
without invokingsuper()
.Parameters: args – Invoked by start()
withextra_args
.By default, screams about using sub-cmds, or about doing nothing!
-
-
class
polyvers.cli.
PolyversCmd
(**kwargs)[source] Bump independently PEP-440 versions of sub-project in Git monorepos.
- SYNTAX:
- {cmd_chain} <sub-cmd> …
-
bootstrapp_projects
() → None[source] Ensure valid configuration exist for monorepo/mono-project(s).
Raises: CmdException – if cwd not inside a git repo
-
collect_app_infos
()[source] Provide extra infos to config infos subcommand.
-
parse_command_line
(argv=None)[source] Parse the command line arguments.
-
class
polyvers.cli.
StatusCmd
(*args, **kw)[source] List the versions of project(s).
- SYNTAX:
- {cmd_chain} [OPTIONS] [<project>]…
-
run
(*pnames)[source] Leaf sub-commands must inherit this instead of
start()
without invokingsuper()
.Parameters: args – Invoked by start()
withextra_args
.By default, screams about using sub-cmds, or about doing nothing!
-
polyvers.cli.
merge_dict
(dct, merge_dct)[source] Recursive dict merge. Inspired by :meth:
dict.update()
, instead of updating only top-level keys, dict_merge recurses down into dicts nested to an arbitrary depth, updating keys. Themerge_dct
is merged intodct
. :param dct: dict onto which the merge is executed :param merge_dct: dct merged into dct :return: NoneAdapted from: https://gist.github.com/angstwad/bf22d1822c38a92ec0a9
-
polyvers.cli.
run
(argv=(), cmd_consumer=None, **app_init_kwds)[source] Handle some exceptions politely and return the exit-code.
Parameters: - argv – Cmd-line arguments, nothing assumed if nohing given.
- cmd_consumer – Specify a different main-mup,
mpu.PrintConsumer
by default. Seempu.pump_cmd()
.
5.2.3. Module: polyvers.pvproject
¶
The main structures of polyvers:
Project 1-->* Engrave 1-->* Graft
-
class
polyvers.pvproject.
Graft
(**kwargs)[source]¶ Instructions on how to search’n replace some text.
-
class
polyvers.pvproject.
Project
(**kwargs)[source]¶ Configurations for projects, in general, and specifically for each one.
-
git_describe
(*git_args, include_lightweight=False, is_release=None, **git_flags)[source]¶ Gets sub-project’s version as derived from
git describe
on its pvtag.Parameters: - include_lightweight – Consider also non-annotated tags when derriving description;
equivalent to
git describe --tags
flag. - is_release –
a 3-state boolean used as index into
tag_vprefixes
:- false: v-tags searched;
- true: r-tags searched;
- None: both tags searched.
- git_args – CLI options passed to
git describe
command. Seeoscmd.PopenCmd
on how to specify cli options using python functions, e.g.('*-v*', '*-r*')
- git_flags –
CLI flags passed to
git describe
command.- See
oscmd.PopenCmd
on how to specify cli flags using python functions, e.g.(all=True)
. - See https://git-scm.com/docs/git-describe
- See
Returns: the pvtag of the current project, or raise
Raises: - GitVoidError – if sub-project is not pvtagged.
- NoGitRepoError – if CWD not within a git repo.
- sbp.CalledProcessError – for any other error while executing git.
Tip
There is also the python==2.7 & python<3.6 safe
polyvers.polyversion()`()
for extracting just the version part from a pvtag; use this one from within project sources.Tip
Same results can be retrieved by this git command:
git describe --tags --match <PROJECT>-v*
where
--tags
is needed to consider also non-annotated tags, asgit tag
does.- include_lightweight – Consider also non-annotated tags when derriving description;
equivalent to
-
last_commit_tstamp
()[source]¶ Report the timestamp of the last commit of the git repo.
Returns: last commit’s timestamp in RFC 2822 format
Raises: - GitVoidError – if there arn’t any commits yet or CWD not within a git repo.
- sbp.CalledProcessError – for any other error while executing git.
Note
Same results can be retrieved by this git command:
git describe --tags --match <PROJECT>-v*
where
--tags
is needed to consider also unannotated tags, asgit tag
does.
Return the full pvtag history for the project, if any.
Raises: AssertionError – If used before populate_pvtags_history()
applied on this project.
-
tag_fnmatch
(is_release=False)[source]¶ The glob-pattern finding pvtags with
git describe --match <pattern>
cmd.Parameters: is_release – False for version-tags, True for release-tags By default, it is interpolated with two PEP 3101 parameters:
{pname} <-- this Project.pname {version} <-- '*'
-
tag_regex
(is_release=False) → Pattern[~AnyStr][source]¶ Interpolate and compile as regex.
Parameters: is_release – False for version-tags, True for release-tags
-
5.2.4. Module: polyvers.pvtags
¶
Git code to make/inspect sub-project “(p)vtags” and respective commits in (mono)repos.
There are 3 important methods/functions calling Git:
Project.git_describe()
that fetches the same version-id thatpolyversion.polyversion()
would return, but with more options.Project.last_commit_tstamp()
, same as above.populate_pvtags_history()
that populates pvtags on the given project instances; certain pvtag-related Project methods would fail if this function has not been applies on a project instance.
A (maybe benign) git-related error
Sub-project has not yet been version with a pvtag.
Command needs a git repo in CWD.
Report pname involved to the user in case tags are missing.
Restored checked out branch to previous state in case of errors (or if forced).
Parameters: restore – if true, force restore at exit, otherwise, restore only on errors
Make a
Project
capturing any pvtag.Useful as a “catch-all” last project in
populate_pvtags_history()
, to capture pvtags not captured by any other project.
Make a
Project
capturing any simple vtag (e.g.v0.1.0
).Useful as a “catch-all” last project in
populate_pvtags_history()
, to capture vtags not captured by any other project.
Make a
Project
for a subprojects hosted at git monorepos.- Project versioned with pvtags like
foo-project-v0.1.0
.
- Project versioned with pvtags like
Make a
Project
for a single project hosted at git repos root (not “monorepos”).- Project versioned with tags simple vtags (not pvtags) like
v0.1.0
.
- Project versioned with tags simple vtags (not pvtags) like
Updates
pvtags_history
on given projects (if any) in ascending order.Parameters: - projects – the projects to search pvtags for
- include_lightweight – fetch also non annotated tags; note that by default,
git-describe
does consider lightweight tags unless--tags
given. - is_release – False for version-tags, True for release-tags
Raises: sbp.CalledProcessError – if git executable not in PATH
Note
Internally, pvtags are populated in
_pvtags_collected
which by default it isNone
. After this call, it will be a (possibly empty) list. Any pre-existing pvtags are removed from all projects before collecting them anew.Tip
To collect all pvtags OR vtags only, use pre-defined projects generated by
make_project_matching_*()
functions.
5.2.5. Module: polyvers.engrave
¶
Search and replace version-ids in files.
-
polyvers.engrave.
glob_files
(patterns: List[str], mybase: Union[str, pathlib.Path] = '.', other_bases: Sequence[Union[str, pathlib.Path]] = None) → List[pathlib.Path][source]¶ Glob files in mybase but not in other_bases (unless bases coincide).
- Supports exclude patterns:
!foo
. - If mybase is in other_bases, it doesn’t change the results.
- Supports exclude patterns:
5.2.6. Module: polyvers.vermath
¶
Validate absolute versions or add relative ones on top of a base absolute.
5.3. Cmdlets¶
5.3.1. Module: polyvers.cmdlet.cfgcmd
¶
Commands to inspect configurations and other cli infos.
-
class
polyvers.cmdlet.cfgcmd.
ConfigCmd
(**kwds)[source]¶ Commands to manage configurations and other cli infos.
-
class
polyvers.cmdlet.cfgcmd.
DescCmd
(**kwds)[source]¶ List and print help for configurable classes and parameters.
- SYNTAX
- {cmd_chain} [-l] [-c] [-t] [-v] [<search-term> …]
- If no search-terms provided, returns all.
- Search-terms are matched case-insensitively against ‘<class>.<param>’, or against ‘<class>’ if –class.
- Use –verbose (-v) to view config-params from the whole hierarchy, that is, including those from intermediate classes.
- Use –class (-c) to view just the help-text of classes.
- Results are sorted in “application order” (later configurations override previous ones); use –sort for alphabetical order.
-
class
polyvers.cmdlet.cfgcmd.
InfosCmd
(**kwargs)[source]¶ List paths and other intallation infos.
- Some of the environment-variables affecting configurations:
- HOME, USERPROFILE, : where configs & DICE projects are stored
- (1st one defined wins)
<APPNAME>_CONFIG_PATHS : where to read configuration-files.
-
class
polyvers.cmdlet.cfgcmd.
ShowCmd
(**kwds)[source]¶ Print configurations (defaults | files | merged) before any validations.
- SYNTAX
- {cmd_chain} [OPTIONS] [–source=(merged | default)] [<search-term-1> …] {cmd_chain} [OPTIONS] –source file
- Search-terms are matched case-insensitively against ‘<class>.<param>’.
- Use –verbose to view values for config-params as they apply in the whole hierarchy (not
- Results are sorted in “application order” (later configurations override previous ones); use –sort for alphabetical order.
- Warning: Defaults/merged might not be always accurate!
- Tip: you may also add –show-config global option on any command to view configured values accurately on runtime.
5.3.2. Module: polyvers.cmdlet.cmdlets
¶
Utils for building elaborate Commands/Sub-commands with traitlets [1] Application.
## Examples:
To run a base command, use this code:
cd = MainCmd.make_cmd(argv, **app_init_kwds) ## `sys.argv` used if `argv` is `None`!
cmd.start()
To run nested commands and print its output, use baseapp.chain_cmds()
like that:
cmd = chain_cmds([MainCmd, Sub1Cmd, Sub2Cmd], argv) ## `argv` without sub-cmds
sys.exit(baseapp.pump_cmd(cmd.start()) and 0)
Of course you can mix’n match.
## Configuration and Initialization guidelines for Spec and Cmd classes
- The configuration of
HasTraits
instance gets stored in itsconfig
attribute. - A
HasTraits
instance receives its configuration from 3 sources, in this order:
- code specifying class-properties or running on constructors;
- configuration files (json or
.py
files);- command-line arguments.
- Constructors must allow for properties to be overwritten on construction; any class-defaults
must function as defaults for any constructor
**kwds
. - Some utility code depends on trait-defaults (i.e. construction of help-messages), so for certain properties (e.g. description), it is preferable to set them as traits-with-defaults on class-properties.
- Listen Good Bait after 1:43.
[1] | (1, 2) http://traitlets.readthedocs.io/ |
-
class
polyvers.cmdlet.cmdlets.
CfgFilesRegistry
(supported_cfg_extensions=['.json', '.py'])[source]¶ Locate and account extensioned files (by default
.json|.py
).- Collects a Locate and (
.json|.py
) files present in the path_list, or - Invoke this for every “manually” visited config-file, successful or not.
- Files collected earlier should override next ones.
-
collect_fpaths
(path_list)[source]¶ Collects all (
.json|.py
) files present in the path_list, (descending order).Parameters: path_list (List[Text]) – A list of paths (absolute, relative, dir or folders). Returns: fully-normalized paths, with ext
-
config_tuples
¶ The consolidated list of loaded 2-tuples
(folder, fname(s))
.Sorted in descending order (1st overrides later).
- Collects a Locate and (
-
class
polyvers.cmdlet.cmdlets.
Cmd
(**kwargs)[source]¶ Common machinery for all (sub)commands.
-
emit_examples
()[source]¶ Yield lines with the usage and examples.
This usage string goes at the end of the command line help string and should contain examples of the application’s usage.
-
emit_help_epilogue
(classes=None)[source]¶ Yield the very bottom lines of the help message.
If classes=False (the default), print –help-all msg.
-
initialize
(argv=None)[source]¶ Invoked after __init__() by make_cmd() to apply configs and build subapps.
Parameters: argv – If undefined, they are replaced with sys.argv[1:]
!It parses cl-args before file-configs, to detect sub-commands and update any
config_paths
, then it reads all file-configs, and then re-apply cmd-line configs as overrides (trick copied from jupyter-core).
-
classmethod
instance
(*args, **kwargs)[source]¶ Returns a global instance of this class.
This method create a new instance if none have previously been created and returns a previously created instance is one already exists.
The arguments and keyword arguments passed to this method are passed on to the
__init__()
method of the class upon instantiation.Create a singleton class using instance, and retrieve it:
>>> from traitlets.config.configurable import SingletonConfigurable >>> class Foo(SingletonConfigurable): pass >>> foo = Foo.instance() >>> foo == Foo.instance() True
Create a subclass that is retrived using the base class instance:
>>> class Bar(SingletonConfigurable): pass >>> class Bam(Bar): pass >>> bam = Bam.instance() >>> bam == Bar.instance() True
-
classmethod
make_cmd
(argv=None, **kwargs)[source]¶ Instanciate, initialize and return application.
Parameters: argv – Like initialize()
, if undefined, replaced withsys.argv[1:]
.
-
read_config_files
(config_paths=None)[source]¶ Load
config_paths
and maintainconfig_registry
.Parameters: config_paths – optional paths to override those in config_paths trait, in descending order (1st overrides the rest). Returns: the static_config loaded - Configuration files are read and merged from
.json
and/or.py
files inconfig_paths
. - Adapted from
load_config_file()
&_load_config_files()
but without applying configs on the app, just returning them.
- Configuration files are read and merged from
-
-
class
polyvers.cmdlet.cmdlets.
CmdletsInterpolation
(*args, **kw)[source]¶ Adds cmdlets_map into interp-manager for for help & cmd mechanics.
Client-code may add more dicts in interpolation_context.maps list.
-
class
polyvers.cmdlet.cmdlets.
Forceable
[source]¶ Mixin to facilitate “forcing” actions by ignoring/delaying their errors.
-
errlogged
(*exceptions, token: Union[bool, str] = None, doing=None, raise_immediately=None, warn_log: Callable = None, info_log: Callable = None)[source]¶ A context-man for nesting
ErrLog
instances.See
ErrLog
for other params.The pre-existing errlog is searched in
_current_errlog
attribute.The _current_errlog on entering context, is restored on exit; original is None.
The returned errlog always has its
ErrLog.parent
set to this enforceable.Example of using this method for multiple actions in a loop:
with self.errlogged(OSError, doing="loading X-files", token='fread'): for fpath in file_paths: with self.errlogged(doing="reading '%s'" % fpath): fbytes.append(fpath.read_bytes()) # Any errors collected above, will be raised/WARNed here.
-
is_forced
(token: Union[str, bool] = True)[source]¶ Whether some action ided by token is allowed to go thorugh in case of errors.
Parameters: token – an optional string/bool to search for in
force
according to the following table:token: |NONE | |FALSE|TRUE|"str"| force-element: |-----|----|-----| [], FALSE| X | X | X | TRUE| X | O | X | '*'| X | X | O | "str"| X | X | = |
- Rows above, win; columns to the left win.
- To catch all tokens, use
--force=true, --force='*'
Tip
prefer using it via
ErrLog
contextman.
-
-
class
polyvers.cmdlet.cmdlets.
PathList
(*args, **kwargs)[source]¶ Trait that splits unicode strings on os.pathsep to form a the list of paths.
-
class
polyvers.cmdlet.cmdlets.
Printable
[source]¶ A
HasTraits
mixin makingstr()
returnclass(trait=value, ...)
from traits.Which traits to print are decided is decided by
TraitSelector
, forprintable_traits
class-property andprintable=True
tag.
-
class
polyvers.cmdlet.cmdlets.
Replaceable
[source]¶ A mixin to make
HasTraits
instances clone like namedtupple’sreplace()
.Parameters: changes – a dict of values keyed be their trait-name. Works nicely with read-only traits.
-
class
polyvers.cmdlet.cmdlets.
Spec
(**kwargs)[source]¶ Common properties for all configurables.
-
classmethod
class_get_trait_help
(trait, inst=None, helptext=None)[source]¶ Get the helptext string for a single trait.
Parameters: - inst – If given, it’s current trait values will be used in place of the class default.
- helptext – If not given, uses the help attribute of the current trait.
-
ikeys
(*maps, **kwds) → ContextManager[polyvers.cmdlet.cmdlets.CmdletsInterpolation][source]¶ Temporarily place self before the given maps and kwds in interpolation-cntxt.
- Self has the least priority, kwds the most.
- For params, see
interp.InterpolationContext.interp()
.
Attention
Must use
str.format_map()
when _stub_keys is true; otherwise,format()
will clone all existing keys in a static map.
-
interp
(text: Union[str, NoneType], *maps, **kwds) → Union[str, NoneType][source]¶ Interpolate text with self attributes before maps and kwds given.
Parameters: text – the text to interplate; None/empty returned as is - For params, see
interp.InterpolationContext.interp()
. - Self has the least priority, kwds the most.
- For params, see
-
classmethod
-
polyvers.cmdlet.cmdlets.
build_sub_cmds
(*subapp_classes)[source]¶ Builds an ordered-dictionary of
cmd-name --> (cmd-class, help-msg)
.
-
polyvers.cmdlet.cmdlets.
chain_cmds
(app_classes, argv=None, **root_kwds)[source]¶ Instantiate a list of
[cmd, subcmd, ...]
, linking children to parents.Parameters: - app_classes – A list of cmd-classes:
[root, sub1, sub2, app]
Note: you have to “know” the correct nesting-order of the commands ;-) - argv – cmdline args are passed to all cmds; make sure they do not contain
any sub-cmds, or chain will be broken.
Like
initialize()
, if undefined, replaced withsys.argv[1:]
.
Returns: The root(1st) cmd to invoke
Aplication.start()
Apply the
pump_cmd()
or collect_cmd() on the return instance.- Normally argv contain any sub-commands, and it is enough to invoke
initialize(argv)
on the root cmd. This function shortcuts arg-parsing for subcmds with explict cmd-chaining in code. - This functions is the 1st half of
Cmd.launch_instance()
.
- app_classes – A list of cmd-classes:
-
polyvers.cmdlet.cmdlets.
class_config_yaml
(cls, outer_cfg, classes=None, config: polyvers._vendor.traitlets.config.loader.Config = None)[source]¶ Get the config section for this Configurable.
Parameters: - classes (list) – (optional) The list of other classes in the config file, used to reduce redundant help descriptions. If given, only params from these classes reported.
- config – If given, only what is contained there is included in generated yaml, with help-descriptions from classes, only where class default-values differ from the values contained in this dictionary.
-
polyvers.cmdlet.cmdlets.
class_help_description_lines
(app_class)[source]¶ “Note: Reverse doc/description order bc classes do not have dynamic default
_desc()
, below.
-
polyvers.cmdlet.cmdlets.
cmd_line_chain
(cmd)[source]¶ Utility returning the cmd-line(str) that launched a
Cmd
.
-
polyvers.cmdlet.cmdlets.
cmdlets_interpolations
= CmdletsInterpolation({}, {'now': <polyvers.cmdlet.interpctxt.Now object>, 'utcnow': <polyvers.cmdlet.interpctxt.Now object>, 'ikeys': <polyvers.cmdlet.interpctxt._KeysDumper object>}, {'$HOSTNAME': 'build-7601792-project-192541-polyvers', '$APPDIR': '/app', '$HOME': '/home/docs', '$OLDPWD': '/', '$READTHEDOCS': 'True', '$READTHEDOCS_PROJECT': 'polyvers', '$PATH': '/home/docs/checkouts/readthedocs.org/user_builds/polyvers/envs/latest/bin:/home/docs/.pyenv/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/docs/.conda/bin:/home/docs/.pyenv/bin', '$LANG': 'C.UTF-8', '$DEBIAN_FRONTEND': 'noninteractive', '$BIN_PATH': '/home/docs/checkouts/readthedocs.org/user_builds/polyvers/envs/latest/bin', '$READTHEDOCS_VERSION': 'latest', '$PWD': '/home/docs/checkouts/readthedocs.org/user_builds/polyvers/checkouts/latest/docs', '$PYENV_ROOT': '/home/docs/.pyenv'}, {'appname': '<APP>', 'cmd_chain': '<CMD>'})¶ That’s the singleton interp-manager used by all cmdlet configurables.
-
polyvers.cmdlet.cmdlets.
generate_config_file_yaml
(self, classes=None, config: polyvers._vendor.traitlets.config.loader.Config = None)[source]¶ generate default config file from Configurables
Parameters: config – If given, only what is contained there is included in generated yaml, with help-descriptions from classes, only where class default-values differ from the values contained in this dictionary.
5.3.3. Module: polyvers.cmdlet.errlog
¶
Suppress or ignore exceptions collected in a nested contexts.
To get a “nested” ErrLog
instance either use nesterrlog()
, or
call ErrLog.__call__()
on the enclosing one.
- FIXME: possible to have different node-hierarchies in contextvars-nesting!!
- (unless
ErrLog()
constructor is never called)
-
class
polyvers.cmdlet.errlog.
ErrLog
(parent: polyvers.cmdlet.cmdlets.Forceable, *exceptions, token: Union[bool, str, NoneType] = None, doing=None, raise_immediately=None, warn_log: Callable = None, info_log: Callable = None)[source]¶ Collects errors in “stacked” contexts and delays or ignores (“forces”) them.
Note
To nest errlogs, prefer
nesterrlog()
instead of this constructor, or else you must keep a reference on the last enclosing errlog and explicitly callErrLog.__call__()
on it.- Unknown errors (not in exceptions) always bubble up immediately.
- Any “forced” errors are collected and logged in warn_log on context-exit,
forcing is enabled when the token given exists in spec’s
force
attribute. - Non-“forced” errors are either raise_immediately, or raised collectively
in a
CollectedErrors
, on context-exit. - Collected are always logged on DEBUG immediately.
- Instances of this class are callable, and the call will return a clone with provided properties updated.
- A clone is also returned when acquiring the context in a with statement.
Variables: - spec – the spec instance to search in its
Spec.force
for the token - exceptions – the exceptions to delay or forced; others are left to bubble immediately
If none given,
Exception
is assumed. - token –
the
force
token to respect, likeSpec.is_forced()
. Resets on each new instance from__call__()
. Possible values:- false: (default) completely ignore force trait
- collected are just delayed);
- <a string>: “force” if this token is in force trait;
- True: “force” if True is in
force``force
.
- doing –
A description of the running activity for the current stacked-context, in present continuous tense, e.g. “readind X-files”.
Resets on each new instance from
__call__()
. Assuming doing = “having fun”, it may generate one of those 3 error messages:Failed having fun due to: EX ...and collected 21 errors (3 ignored): - Err 1: ... Collected 21 errors (3 ignored) while having fun: - Err 1: ... LOG: Ignored 9 errors while having fun: - Err 1: ...
- raise_immediately – if not forced, do not wait for report() call to raise them;
suggested use when a function decorator. Also when –debug.
Resets on each new instance from
__call__()
. - warn_log – the logging method to report forced errors; if none given, use searched the log ov the parent or falls back to this’s modules log. Note that all failures are always reported immediately on DEBUG.
- info_log – the logging method to report completed “doing” tasks; if none given does not report them.
PRIVATE FIELDS:
Variables: - _root_node – The root of the tree of nodes, populated when entering contexts recursively.
- _anchor – the parent node in the tree where on
__enter__()
a new _active child-node is attached, and the tree grows. - _active – the node created in _anchor
Example of using this method for multiple actions in a loop:
with ErrLog(enforeceable, OSError, doing="loading X-files", token='fread') as erl1: for fpath in file_paths: with erl1(doing="reading '%s'" % fpath) as erl2: fbytes.append(fpath.read_bytes()) # Any errors collected will raise/WARN here (root-context exit).
-
coords
¶ Return my anchor’s coordinate-indices from the root of the tree.
-
is_armed
¶ Is context
__enter__()
currently under process?
-
is_forced
¶ Try force in parent first.
-
is_good
¶ An errlog is “good” if its anchor has not captured any exception yet.
If it does, it cannot be used anymore.
-
pdebug
¶ Search debug property in parent, or False.
-
plog
¶ Search log property in parent, or use this module’s logger.
-
report_root
(ex_raised: Union[Exception, NoneType]) → Union[_ForwardRef('CollectedErrors'), NoneType][source]¶ Raise or log the errors collected from
Parameters: ex_raised – the cntxt-body exception __exit__()
is about to raiseReturns: a CollectedErrors
in case catured errors contain non-forced errors BUT ex_raised given.Raises: CollectedErrors – any non-forced exceptions captured in tree (if any), unless ex_raised given
-
polyvers.cmdlet.errlog.
errlogged
(*errlog_args, **errlog_kw)[source]¶ Decorate functions/methods with a
ErrLog
instance.The errlog-contextman is attached on the wrapped function/method as the errlog attribute.
-
polyvers.cmdlet.errlog.
nesterrlog
(parent, *exceptions, token: Union[bool, str] = None, doing=None, raise_immediately=None, warn_log: Callable = None, info_log: Callable = None)[source]¶ To nest errlogs, prefer this function instead of this
ErrLog()
constructor, or else you must keep a reference on the last enclosing errlog and explicitly callErrLog.__call__()
on it.
5.3.4. Module: polyvers.cmdlet.interpctxt
¶
Enable Unicode-trait to pep3101-interpolate {key} patterns from “context” dicts.
-
class
polyvers.cmdlet.interpctxt.
InterpolationContext
[source]¶ A stack of 4 dics to be used as interpolation context.
- The 3 stacked dicts are:
- user-map: writes affect permanently this dict only;
- time: (‘now’, ‘utcnow’), always updated on access;
- env-vars, $-prefixed.
Append more dicts in
self.maps
list if you wish.-
ikeys
(*maps, _stub_keys: Union[str, bool, NoneType] = False, _escaped_for: Union[typing.Callable, str] = None, **kv_pairs) → ContextManager[_ForwardRef('InterpolationContext')][source]¶ Temporarily place maps and kwds immediately after user-map (2nd position).
- For params, see
interp()
.
Attention
Must use
str.format_map()
when _stub_keys is true; otherwise,format()
will clone all existing keys in a static map.- For params, see
-
interp
(text: Union[str, NoneType], *maps, _stub_keys=False, _escaped_for: Union[typing.Callable, str] = None, _suppress_errors: bool = None, **kv_pairs) → Union[str, NoneType][source]¶ Interpolate text with values from maps and kwds given.
Parameters: - text – the text to interpolate; if null/empty, returned as is
- maps – a list of dictionaries/objects/HasTraits from which to draw items/attributes/trait-values, all in increasing priority. Nulls ignored.
- _stub_keys –
- If false, missing keys raise KeyError.
- If True, any missing key gets replaced by
{key}
(practically remain unchanged). - If callable, the key is passed to it as a the only arg, and the result gets replaced.
- Any other non-false value is returned for every key.
- _suppress_errors – ignore any interpolation errors and return original string
- _escaped_for – a callable or (‘glob’|’regex’) to escape object’s attribute values
Later maps take precedence over earlier ones; kv_pairs have the highest, but _stub_keys the lowest (if true).
-
polyvers.cmdlet.interpctxt.
dictize_object
(obj, _escaped_for: Union[typing.Callable, str] = None)[source]¶ Make an object appear as a dict for
InterpolationContext.ikeys()
.Parameters: _escaped_for – one of ‘glob’, ‘regex’ or a callable to escape object’s attribute values
5.4. Utilities¶
5.4.1. Module: polyvers.utils.mainpump
¶
Utils pumping results out of yielding functions for main().
-
class
polyvers.utils.mainpump.
ConsumerBase
[source]¶ Checks if all boolean items (if any) are True, to decide final bool state.
-
class
polyvers.utils.mainpump.
ListConsumer
[source]¶ Collect all items in a list, while checking if all boolean ok.
-
class
polyvers.utils.mainpump.
PrintConsumer
[source]¶ Prints any text-items while checking if all boolean ok.
-
polyvers.utils.mainpump.
collect_cmd
(cmd_res, dont_coalesce=False, assert_ok=False)[source]¶ Pumps cmd-result in a new list.
Parameters: - cmd_res – A list of items returned by a
Cmd.start()
/Cmd.run(). If it is a sole item, it is returned alone without a list. - assert_ok – if true, checks
ListConsumer
’s exit-code is not false.
- cmd_res – A list of items returned by a
-
polyvers.utils.mainpump.
pump_cmd
(cmd_res, consumer=None)[source]¶ Sends (possibly lazy) cmd-results to a consumer (by default to STDOUT).
Parameters: - cmd_res – Whatever is returnened by a
Cmd.start()
/Cmd.run(). - consumer – A callable consuming items and deciding if everything was ok;
defaults to
PrintConsumer
Returns: bool(consumer)
- Remember to have logging setup properly before invoking this.
- This the 2nd half of the replacement for
Application.launch_instance()
.
- cmd_res – Whatever is returnened by a
5.4.2. Module: polyvers.utils.logconfutils
¶
Utils for configuring and using elaborate logs and handling main() failures.
-
polyvers.utils.logconfutils.
exit_with_pride
(reason=None, warn_color='\x1b[31;1m', err_color='\x1b[1m', logger=None)[source]¶ Return an exit-code and logs error/fatal message for
main()
methods.Parameters: - reason –
- If reason is None, exit-code(0) signifying OK;
- if exception, print colorful (if tty) stack-trace, and exit-code(-1);
- otherwise, prints str(reason) colorfully (if tty) and exit-code(1),
- warn_color – ansi color sequence for stack-trace (default: red)
- err_color – ansi color sequence for stack-trace (default: white-on-red)
- logger – Which logger to use to log reason (must support info and fatal). if missing, derived from this module.
Returns: (0, 1 -1), for reason == (None, str, Exception) respectively.
Note that returned string from
main()
are printed to stderr and exit-code set to bool(str) = 1, so print stderr separately and then set the exit-code.For colors use
RainbowLoggingHandler.getColor()
, defaults: - ‘[33;1m’: yellow+bold - ‘[31;1m’: red+boldNote: it’s better to have initialized logging.
- reason –
-
polyvers.utils.logconfutils.
init_logging
(level=20, logconf=None, color=None, logger=None, **kwds)[source]¶ Parameters: - level – Root-logger’s level; Overrides logconf if given, INFO otherwise.
- logconf (None, str, seq[str]) –
File(s) to configure loggers; set [] to prohibit loading any logconf file. Allowed file-extensions:
- ’.conf’ (implied if missing) .
- ’.yml’/’yaml’
The ~ in the path expanded to $HOME. See https://docs.python.org/3/library/logging.config.html
- color – Whether to color log-messages; if undefined, true only in consoles.
- logger – Which logger to use to log logconf source(must support info and debug). if missing, derived from this module.
- kwds – Passed directly to
logging.basicConfig()
(e.g. filename); used only id default HOMElogconf.yaml
file is NOT read.
5.4.3. Module: polyvers.utils.oscmd
¶
Utility to call OS commands through subprocess.run()
with logging.
The polyvers version-configuration tool is generating tags like:
proj-foo-v0.1.0
On purpose python code here kept with as few dependencies as possible.
-
class
polyvers.utils.oscmd.
PopenCmd
(dry_run=False, check_stdout=True, check_stderr=True, check_returncode=True, **popen_kw)[source]¶ A function –> cmd-line builder for executing (mostly) git commands.
To run
git log -n1
:out = cmd.git.log(n=1)
To launch a short python program with
python -c "print('a')"
:out = cmd.python._(c=True)('print('a')')
Raises: sbp.CalledProcessError – if check_returncode=true and exit code is non-zero. Important
Flags written out are mostly for Git, bc flags are produced like that:
-f <value> --flag=<value>
-
polyvers.utils.oscmd.
exec_cmd
(cmd, dry_run=False, check_stdout=True, check_stderr=True, check_returncode=True, encoding='utf-8', encoding_errors='surrogateescape', **popen_kws)[source]¶ Parameters: check_stdout – None: Popen(stdout=None), printed False: Popen(stdout=sbp.DEVNULL), ignored True: Popen(stdout=sbp.PIPE), collected & returned
5.4.4. Module: polyvers.utils.fileutil
¶
Generic utils.
-
polyvers.utils.fileutil.
convpath
(fpath, abs_path=True, exp_user=True, exp_vars=True)[source]¶ Without any flags, just pass through
osp.normpath()
.
-
polyvers.utils.fileutil.
ensure_dir_exists
(path, mode=493)[source]¶ ensure that a directory exists
If it doesn’t exist, try to create it and protect against a race condition if another process is doing the same.
The default permissions are 755, which differ from os.makedirs default of 777.
-
polyvers.utils.fileutil.
ensure_file_ext
(fname, ext, *exts, is_regex=False)[source]¶ Ensure that the filepath ends with the extension(s) specified.
Parameters: - ext (str) – The 1st extension to search & to append if none matches, so must not be a regex.
- exts (str) – Other extensions. These may be regexes, depending on is_regex; a ‘$’ is always added at its end.
- is_regex (bool) – When true, the rest exts are parsed as case-insensitive regexes.
Example:
>>> ensure_file_ext('foo', '.bar') 'foo.bar' >>> ensure_file_ext('foo.', '.bar') 'foo.bar' >>> ensure_file_ext('foo.', 'bar') 'foo.bar' >>> ensure_file_ext('foo.BAR', '.bar') 'foo.BAR' >>> ensure_file_ext('foo.DDD', '.bar') 'foo.DDD.bar'
Note that omitting dot(‘.’) from extension does affect the results:
>>> ensure_file_ext('foo', 'bar') 'foo.bar' >>> ensure_file_ext('foo.BAR', 'bar') 'foo.BAR' >>> ensure_file_ext('fooBAR', 'bar') # File allowed without extension! 'fooBAR'
When more extensions are given, the 1st is appended if none matches:
>>> ensure_file_ext('foo.xlt', '.xlsx', '.XLT') 'foo.xlt' >>> ensure_file_ext('foo.xlt', '.xlsx', '.xltx') 'foo.xlt.xlsx'
And when regexes:
>>> ensure_file_ext('foo.xlt', '.xlsx', r'\.xl\w{1,2}', is_regex=True) 'foo.xlt' >>> ensure_file_ext('foo.xl^', '.xls', r'\.xl\w{1,2}', is_regex=True) 'foo.xl^.xls'