swiftly.cli¶

Contains swiftly.cli.cli.CLI that implements the swiftly command line interface and all the surrounding implementation modules.

CLI is imported here for convenience.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

cli¶

Contains the CLI class that handles the original command line.

class swiftly.cli.cli.CLI[source]¶

Bases: object

Handles the original command line.

An example script is swiftly itself:

#!/usr/bin/env python
import sys
import swiftly.cli
sys.exit(swiftly.cli.CLI()())

See the output of swiftly help for more information.

commands = None¶: A dictionary of the available commands and their CLICommand instances.

context = None¶

The overall CLIContext containing attributes generated by the initial main_options parsing.

The available attributes are:

cdn	True if the CDN URL should be used instead of the default Storage URL.
client_manager	The `swiftly.client.manager.ClientManager` to use for obtaining clients.
concurrency	Number of concurrent actions to allow.
io_manager	The `swiftly.cli.iomanager.IOManager` to use for input and output.
eventlet	True if Eventlet is in use.
original_args	The original args used by the CLI.
original_begin	The original time.time() when the CLI was called.
verbose	Function to call when you want to (optionally) emit verbose output. `verbose(msg, *args)` where the output will be constructed with `msg % args`.
verbosity	Level of verbosity. Just None or 1 right now.

option_parser = None¶: The main OptionParser.

swiftly.cli.cli.COMMANDS = ['swiftly.cli.auth.CLIAuth', 'swiftly.cli.decrypt.CLIDecrypt', 'swiftly.cli.delete.CLIDelete', 'swiftly.cli.encrypt.CLIEncrypt', 'swiftly.cli.fordo.CLIForDo', 'swiftly.cli.get.CLIGet', 'swiftly.cli.head.CLIHead', 'swiftly.cli.help.CLIHelp', 'swiftly.cli.ping.CLIPing', 'swiftly.cli.post.CLIPost', 'swiftly.cli.put.CLIPut', 'swiftly.cli.tempurl.CLITempURL', 'swiftly.cli.trans.CLITrans']¶: The list of CLICommand classes avaiable to CLI. You’ll want to add any new CLICommand you create to this list.

context¶

Contains the CLIContext class used to pass contextual information to CLI functions.

class swiftly.cli.context.CLIContext[source]¶

Bases: object

Used to pass contextual information to CLI functions.

An instance of this class will allow almost any attribute access, even if that attribute did not previously exist. It offers an easy way to set new attributes on the fly for different use cases without having to create a whole new subclasses.

copy()[source]¶: Returns a new CLIContext instance that is a shallow copy of the original, much like dict’s copy method.

write_headers(fp, headers, mute=None)[source]¶: Convenience function to output headers in a formatted fashion to a file-like fp, optionally muting any headers in the mute list.

command¶

Contains the CLICommand class that can be subclassed to create new Swiftly commands.

class swiftly.cli.command.CLICommand(cli, name, min_args=None, max_args=None, usage=None)[source]¶

Bases: object

Subclass this to create new Swiftly commands.

Don’t forget to add your new class to swiftly.cli.cli.COMMANDS.

Your subclass will be created by swiftly.cli.cli.CLI with the CLI instance as the only parameter. You should then call this superclass with that CLI instance, your command’s name, and any other options desired.

See the implemention for other commands like swiftly.cli.auth.CLIAuth for good starting points.

Parameters:	cli – The `swiftly.cli.cli.CLI` instance. name – The name of the command. min_args – The minimum number of arguments required. max_args – The maximum number of arguments allowed. usage – The usage string for `OptionParser`.

options_list_to_lowered_dict(options_list)[source]¶

Helper function that will convert an options list into a dict of key/values.

This is used for the quite common -hheader:value and -qparameter=value command line options, like this:

context.headers = self.options_list_to_lowered_dict(options.header)
context.query = self.options_list_to_lowered_dict(options.query)

For a full example, see swiftly.cli.get.CLIGet.__call__().

parse_args_and_create_context(args)[source]¶

Helper method that will parse the args into options and remaining args as well as create an initial swiftly.cli.context.CLIContext.

The new context will be a copy of swiftly.cli.cli.CLI.context with the following attributes added:

muted_account_headers	The headers to omit when outputting account headers.
muted_container_headers	The headers to omit when outputting container headers.
muted_object_headers	The headers to omit when outputting object headers.

Returns:	options, args, context

exception swiftly.cli.command.ReturnCode(text=None, code=1)[source]¶

Bases: exceptions.Exception

Raise this to indicate the desire to exit.

Parameters:	text – The text to report as the reason for exiting. Default: None code – The return code to give back to the shell. Default: 1

optionparser¶

Contains an extended optparse.OptionParser that works better with command line applications that may call other command line applications or use multiple parsers.

class swiftly.cli.optionparser.OptionParser(usage=None, option_list=None, option_class=<class optparse.Option at 0x23f93b0>, version=None, conflict_handler='error', description=None, formatter=None, add_help_option=True, prog=None, epilog=None, io_manager=None, raw_epilog='', error_prefix='')[source]¶

Bases: optparse.OptionParser, object

Extended optparse.OptionParser that supports an io_manager, raw_epilog output, and an error prefix. It also does not exit automatically on error and instead sets an instance bool of error_encountered.

error(msg, file=None)[source]¶: Outputs the error msg to the file if specified, or to the io_manager’s stderr if available, or to sys.stderr.

error_encountered = None¶: True if an error was encountered while parsing.

error_prefix = None¶: Output just before any error. This can be useful in identification when multiple parsers are in use.

exit(status=0, msg=None)[source]¶: Immediately exits Python with the given status (or 0) as the exit code and optionally outputs the msg using self.error.

print_help(file=None)[source]¶: Outputs help information to the file if specified, or to the io_manager’s stdout if available, or to sys.stdout.

print_usage(file=None)[source]¶: Outputs usage information to the file if specified, or to the io_manager’s stdout if available, or to sys.stdout.

print_version(file=None)[source]¶: Outputs version information to the file if specified, or to the io_manager’s stdout if available, or to sys.stdout.

raw_epilog = None¶: Output just after the standard print_help output, in it’s raw form. This is different than epilog in that epilog is reformatted.

iomanager¶

Contains IOManager for managing access to input, output, error, and debug file-like objects.

class swiftly.cli.iomanager.IOManager(stdin=None, stdout=None, stderr=None, debug=None, stdin_root=None, stdout_root=None, stderr_root=None, debug_root=None, stdin_sub_command=None, stdout_sub_command=None, stderr_sub_command=None, debug_sub_command=None, subprocess_module=None, verbose=None)[source]¶

Bases: object

Manages access to IO in ways that are mostly specific to swiftly.cli but might be generally useful.

Parameters:

stdin – The file-like object to use for default stdin or sys.stdin if None.
stdout – The file-like object to use for default stdout or sys.stdout if None.
stderr – The file-like object to use for default stderr or sys.stderr if None.
debug – The file-like object to use for default debug output or sys.stderr if None.
stdin_root – The root path to use for requests for pathed stdin file-like objects.
stdout_root – The root path to use for requests for pathed stdout file-like objects.
stderr_root – The root path to use for requests for pathed stderr file-like objects.
debug_root – The root path to use for requests for pathed debug file-like objects.
stdin_sub_command – A shell line to pipe any stdin file-like objects through.
stdout_sub_command – A shell line to pipe any stdout file-like objects through.
stderr_sub_command – A shell line to pipe any stderr file-like objects through.
debug_sub_command – A shell line to pipe any debug file-like objects through.
subprocess_module – The subprocess module to use; for instance, you might use eventlet.green.subprocess instead of the standard subprocess module.
verbose – A function to call with (msg) when waiting for subcommands to complete and for logging the subcommands’ return codes.

client_path_to_os_path(client_path)[source]¶

Converts a client path into the operating system’s path by replacing instances of ‘/’ with os.path.sep.

Note: If the client path contains any instances of os.path.sep already, they will be replaced with ‘-‘.

get_debug(os_path=None, skip_sub_command=False)[source]¶: Returns a debug-output-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

get_stderr(os_path=None, skip_sub_command=False)[source]¶: Returns a stderr-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

get_stdin(os_path=None, skip_sub_command=False)[source]¶: Returns a stdin-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

get_stdout(os_path=None, skip_sub_command=False)[source]¶: Returns a stdout-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

os_path_to_client_path(os_path)[source]¶

Converts an operating system path into a client path by replacing instances of os.path.sep with ‘/’.

Note: If the client path contains any instances of ‘/’ already, they will be replaced with ‘-‘.

with_debug(*args, **kwds)[source]¶

A context manager yielding a debug-output-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

Parameters:	os_path – Optional path to base the file-like object on. skip_sub_command – Set True to skip any configured sub-command filter. disk_closed_callback – If the backing of the file-like object is an actual file that will be closed, disk_closed_callback (if set) will be called with the on-disk path just after closing it.

with_stderr(*args, **kwds)[source]¶

A context manager yielding a stderr-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

Parameters:	os_path – Optional path to base the file-like object on. skip_sub_command – Set True to skip any configured sub-command filter. disk_closed_callback – If the backing of the file-like object is an actual file that will be closed, disk_closed_callback (if set) will be called with the on-disk path just after closing it.

with_stdin(*args, **kwds)[source]¶

A context manager yielding a stdin-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

Parameters:	os_path – Optional path to base the file-like object on. skip_sub_command – Set True to skip any configured sub-command filter. disk_closed_callback – If the backing of the file-like object is an actual file that will be closed, disk_closed_callback (if set) will be called with the on-disk path just after closing it.

with_stdout(*args, **kwds)[source]¶

A context manager yielding a stdout-suitable file-like object based on the optional os_path and optionally skipping any configured sub-command.

Parameters:	os_path – Optional path to base the file-like object on. skip_sub_command – Set True to skip any configured sub-command filter. disk_closed_callback – If the backing of the file-like object is an actual file that will be closed, disk_closed_callback (if set) will be called with the on-disk path just after closing it.

auth¶

Contains a CLICommand that authenticates and then outputs the resulting information.

Uses the following from swiftly.cli.context.CLIContext:

client_manager	For connecting to Swift.
io_manager	For directing output.

class swiftly.cli.auth.CLIAuth(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that authenticates and then outputs the resulting information.

See the output of swiftly help auth for more information.

swiftly.cli.auth.cli_auth(context)[source]¶

Authenticates and then outputs the resulting information.

See swiftly.cli.auth for context usage information.

See CLIAuth for more information.

decrypt¶

Contains a CLICommand for decrypting stdin to stdout.

Uses the following from swiftly.cli.context.CLIContext:

io_manager

For directing output.

class swiftly.cli.decrypt.CLIDecrypt(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand for decrypting stdin and sending that to stdout.

See the output of swiftly help decrypt for more information.

swiftly.cli.decrypt.cli_decrypt(context, key)[source]¶

Decrypts context.io_manager’s stdin and sends that to context.io_manager’s stdout.

See swiftly.cli.decrypt for context usage information.

See CLIDecrypt for more information.

delete¶

Contains a CLICommand that can issue DELETE requests.

Uses the following from swiftly.cli.context.CLIContext:

cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
concurrency	The number of concurrent actions that can be performed.
headers	A dict of headers to send.
ignore_404	True if 404s should be silently ignored.
io_manager	For directing output.
query	A dict of query parameters to send.

class swiftly.cli.delete.CLIDelete(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue DELETE requests.

See the output of swiftly help delete for more information.

swiftly.cli.delete.cli_delete(context, path, body=None, recursive=False, yes_empty_account=False, yes_delete_account=False)[source]¶

Deletes the item (account, container, or object) at the path.

See swiftly.cli.delete for context usage information.

See CLIDelete for more information.

Parameters:

context – The swiftly.cli.context.CLIContext to use.
path – The path of the item (acount, container, or object) to delete.
body – The body to send with the DELETE request. Bodies are not normally sent with DELETE requests, but this can be useful with bulk deletes for instance.
recursive – If True and the item is an account or container, deletes will be issued for any containing items as well.
yes_empty_account – This must be set to True for verification when the item is an account and recursive is True.
yes_delete_account – This must be set to True for verification when the item is an account and you really wish a delete to be issued for the account itself.

swiftly.cli.delete.cli_empty_account(context, yes_empty_account=False)[source]¶

Empties the account of all objects and containers; you must set yes_empty_account to True to verify you really want to do this.

See swiftly.cli.delete for context usage information.

See CLIDelete for more information.

swiftly.cli.delete.cli_empty_container(context, path)[source]¶

Empties the container at the path of all objects.

See swiftly.cli.delete for context usage information.

See CLIDelete for more information.

encrypt¶

Contains a CLICommand for encrypting stdin to stdout.

Uses the following from swiftly.cli.context.CLIContext:

io_manager

For directing output.

class swiftly.cli.encrypt.CLIEncrypt(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand for encrypting stdin and sending that to stdout.

This can be useful to encrypt to disk before attempting to upload, allowing uploads retries and segmented encrypted objects.

See the output of swiftly help encrypt for more information.

swiftly.cli.encrypt.cli_encrypt(context, key)[source]¶

Encrypts context.io_manager’s stdin and sends that to context.io_manager’s stdout.

This can be useful to encrypt to disk before attempting to upload, allowing uploads retries and segmented encrypted objects.

See swiftly.cli.encrypt for context usage information.

See CLIEncrypt for more information.

fordo¶

Contains a CLICommand that can issue other commands for each item in an account or container listing.

Uses the following from swiftly.cli.context.CLIContext:

cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
concurrency	The number of concurrent actions that can be performed.
headers	A dict of headers to send.
ignore_404	True if 404s should be silently ignored.
io_manager	For directing output.
query	A dict of query parameters to send. Of important use are limit, delimiter, prefix, marker, and end_marker as they are common listing query parameters.
remaining_args	The list of command line args to issue to the sub-CLI instance; the first arg that equals ‘<item>’ will be replaced with each item the for encounters. Any additional instances of ‘<item>’ will be left alone, as you might be calling a nested “for ... do”.
original_main_args	Used when constructing sub-CLI instances.
output_names	If True, outputs the name of each item just before calling [command] with it. To ensure easier parsing, the name will be url encoded and prefixed with “Item Name: ”. For commands that have output of their own, this is usually only useful with single concurrency; otherwise the item names and the command output will get interspersed and impossible to associate.

class swiftly.cli.fordo.CLIForDo(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue other commands for each item in an account or container listing.

See the output of swiftly help for for more information.

swiftly.cli.fordo.cli_fordo(context, path=None)[source]¶

Issues commands for each item in an account or container listing.

See swiftly.cli.fordo for context usage information.

See CLIForDo for more information.

get¶

Contains a CLICommand that can issue GET requests.

Uses the following from swiftly.cli.context.CLIContext:

all_objects	True if instead of the listing itself you want all objects that the listing references to be output.
cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
concurrency	The number of concurrent actions that can be performed.
full	True if you want a full listing (additional information like object count, bytes used, and upload date) instead of just the item names.
headers	A dict of headers to send.
ignore_404	True if 404s should be silently ignored.
io_manager	For directing output.
muted_account_headers	The headers to omit when outputting account response headers.
muted_container_headers	The headers to omit when outputting container response headers.
muted_object_headers	The headers to omit when outputting object response headers.
output_headers	True if you want the headers from the response to also be output.
query	A dict of query parameters to send. Of important use are limit, delimiter, prefix, marker, and end_marker as they are common listing query parameters.
raw	Normally the result of the GET is translated as a listing and formatted output is generated. Setting this to True will skip any formatting a just output the raw contents from the response. Note that this will also just issue a single request and will not try additional follow-on marker requests.
remove_empty_files	True if files created on disk should be removed if they result in an empty file. This can be useful with sub_commands that only output information for matches.
suppress_container_name	True if files created on disk should have the container name stripped from the file name. When downloading a single container, this is usually desired.
write_headers	A function used to output the response headers if output_headers is set True.

class swiftly.cli.get.CLIGet(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue GET requests.

See the output of swiftly help get for more information.

swiftly.cli.get.cli_get(context, path=None)[source]¶

Performs a GET on the item (account, container, or object).

See swiftly.cli.get for context usage information.

See CLIGet for more information.

swiftly.cli.get.cli_get_account_listing(context)[source]¶

Performs a GET on the account as a listing request.

See swiftly.cli.get for context usage information.

See CLIGet for more information.

swiftly.cli.get.cli_get_container_listing(context, path=None)[source]¶

Performs a GET on the container as a listing request.

See swiftly.cli.get for context usage information.

See CLIGet for more information.

head¶

Contains a CLICommand that can issue HEAD requests.

Uses the following from swiftly.cli.context.CLIContext:

cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
headers	A dict of headers to send.
ignore_404	True if 404s should be silently ignored.
io_manager	For directing output.
muted_account_headers	The headers to omit when outputting account response headers.
muted_container_headers	The headers to omit when outputting container response headers.
muted_object_headers	The headers to omit when outputting object response headers.
query	A dict of query parameters to send.
write_headers	A function used to output the response headers.

class swiftly.cli.head.CLIHead(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue HEAD requests.

See the output of swiftly help head for more information.

swiftly.cli.head.cli_head(context, path=None)[source]¶

Performs a HEAD on the item (account, container, or object).

See swiftly.cli.head for context usage information.

See CLIHead for more information.

help¶

Contains a CLICommand that outputs help information.

Uses the following from swiftly.cli.context.CLIContext:

io_manager

For directing output.

class swiftly.cli.help.CLIHelp(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that outputs help information.

See the output of swiftly help help for more information.

swiftly.cli.help.cli_help(context, command_name, general_parser, command_parsers)[source]¶

Outputs help information.

See swiftly.cli.help for context usage information.

See CLIHelp for more information.

Parameters:	context – The `swiftly.cli.context.CLIContext` to use. command_name – The command_name to output help information for, or set to None or an empty string to output the general help information. general_parser – The `swiftly.cli.optionparser.OptionParser` for general usage. command_parsers – A dict of (name, `CLICommand`) for specific command usage.

ping¶

Contains a CLICommand that implements ping test functionality.

Uses the following from swiftly.cli.context.CLIContext:

client_manager	For connecting to Swift.
concurrency	The number of concurrent actions that can be performed.
io_manager	For directing output.
limit	The maximum number of Swift nodes to output information about.
object_ring	An instance of swift.common.ring.ring.Ring if you want a report based on Swift nodes with implied usage during the ping test.
ping_begin	The first time.time() when the entire ping test began.
ping_begin_last	The time.time() the last ping task started.
ping_count	The number of objects to use.
ping_verbose	True if you want a full ping report rather than just the overall time.
threshold	Defines the threshold for the threshold node report. This is the multiplier over the average request time.

class swiftly.cli.ping.CLIPing(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that implements ping test functionality.

See the output of swiftly help ping for more information.

swiftly.cli.ping.cli_ping(context, prefix)[source]¶

Performs a ping test.

See swiftly.cli.ping for context usage information.

See CLIPing for more information.

Parameters:	context – The `swiftly.cli.context.CLIContext` to use. prefix – The container name prefix to use. Default: swiftly-ping

post¶

Contains a CLICommand that can issue POST requests.

Uses the following from swiftly.cli.context.CLIContext:

cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
headers	A dict of headers to send.
io_manager	For directing output.
query	A dict of query parameters to send.

class swiftly.cli.post.CLIPost(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue POST requests.

See the output of swiftly help post for more information.

swiftly.cli.post.cli_post(context, path, body=None)[source]¶

Performs a POST on the item (account, container, or object).

See swiftly.cli.post for context usage information.

See CLIPost for more information.

Parameters:	context – The `swiftly.cli.context.CLIContext` to use. path – The path to the item to issue a POST for. body – The body send along with the POST.

put¶

Contains a CLICommand that can issue PUT requests.

Uses the following from swiftly.cli.context.CLIContext:

cdn	True if the CDN Management URL should be used instead of the Storage URL.
client_manager	For connecting to Swift.
concurrency	The number of concurrent actions that can be performed.
different	Set to True to check if the local file is different than an existing object before uploading.
empty	Set to True if you wish to send an empty body with the PUT rather than reading from the io_manager’s stdin.
headers	A dict of headers to send.
input_	A string representing where input should be obtained from. If None, the io_manager’s stdin will be used. If a directory path is specified, a set of PUTs will be generated for each item in the directory structure. If a file path is specified, that single file will be used as input.
io_manager	For directing output and obtaining input if needed.
newer	Set to True to check if the local file is newer than an existing object before uploading.
query	A dict of query parameters to send.
seek	Where to seek to in the input_ before uploading; usually just used by recursive calls with segmented objects.
segment_size	The max size of a file before switching to a segmented object and the max size of each object segment.
static_segments	Set to True to use static large object support instead of dynamic large object support.

class swiftly.cli.put.CLIPut(cli)[source]¶

Bases: swiftly.cli.command.CLICommand

A CLICommand that can issue PUT requests.

See the output of swiftly help put for more information.

swiftly.cli.put.cli_put(context, path)[source]¶

Performs a PUT on the item (account, container, or object).

See swiftly.cli.put for context usage information.