swh.deposit package

Submodules

swh.deposit.apps module

class swh.deposit.apps.DepositConfig(app_name, app_module)[source]

Bases: django.apps.config.AppConfig

name = 'swh.deposit'
ready()[source]

Override this method in subclasses to run code when Django starts.

swh.deposit.auth module

swh.deposit.auth.convert_response(request, content)[source]

Convert response from drf’s basic authentication mechanism to a swh-deposit one.

Args:
request (Request): Use to build the response content (bytes): The drf’s answer

Returns:

Response with the same status error as before, only the body is now an swh-deposit compliant one.
class swh.deposit.auth.WrapBasicAuthenticationResponseMiddleware(get_response)[source]

Bases: object

Middleware to capture potential authentication error and convert them to standard deposit response.

This is to be installed in django’s settings.py module.

swh.deposit.config module

swh.deposit.config.setup_django_for(platform)[source]
Setup function for command line tools (swh.deposit.create_user) to
initialize the needed db access.

Note

Do not import any django related module prior to this function call. Otherwise, this will raise an django.core.exceptions.ImproperlyConfigured error message.

Parameters:platform (str) – the platform the scheduling is running
Raises:ValueError in case of wrong platform inputs.
class swh.deposit.config.SWHDefaultConfig(**config)[source]

Bases: swh.core.config.SWHConfig

Mixin intended to enrich views with SWH configuration.

CONFIG_BASE_FILENAME = 'deposit/server'
DEFAULT_CONFIG = {'checks': ('bool', True), 'max_upload_size': ('int', 209715200), 'scheduler': ('dict', {'args': {'url': 'http://localhost:5008/'}, 'cls': 'remote'})}
ADDITIONAL_CONFIG = {}

swh.deposit.create_user module

swh.deposit.errors module

Module in charge of providing the standard sword errors

swh.deposit.errors.make_error_dict(key, summary=None, verbose_description=None)[source]

Utility function to factorize error message dictionary.

Parameters:
  • key (str) – Error status key referenced in swh.deposit.errors module
  • summary (str/None) – Error message clarifying the status
  • verbose_description (str/None) – A more verbose description or work around a potential problem.
Returns:

Dictionary with key ‘error’ detailing the ‘status’ and associated ‘message’
swh.deposit.errors.make_error_response_from_dict(req, error)[source]

Utility function to return an http response with error detail.

Parameters:
  • req (Request) – original request
  • error (dict) – Error described as dict, typically generated
  • the make_error_dict function. (from) –
Returns:

HttpResponse with detailed error.
swh.deposit.errors.make_error_response(req, key, summary=None, verbose_description=None)[source]

Utility function to create an http response with detailed error.

Parameters:
  • req (Request) – original request
  • key (str) – Error status key referenced in swh.deposit.errors module
  • summary (str) – Error message clarifying the status
  • verbose_description (str / None) – A more verbose description or work around a potential problem.
Returns:

Dictionary with key ‘error’ detailing the ‘status’ and associated ‘message’

swh.deposit.models module

class swh.deposit.models.Dbversion(*args, **kwargs)[source]

Bases: django.db.models.base.Model

Db version

version

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

release

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

description

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

objects = <django.db.models.manager.Manager object>
swh.deposit.models.DEPOSIT_STATUS = [('partial', 'partial'), ('expired', 'expired'), ('deposited', 'deposited'), ('verified', 'verified'), ('rejected', 'rejected'), ('loading', 'loading'), ('done', 'done'), ('failed', 'failed')]

Possible status and the detailed meaning.

class swh.deposit.models.DepositClient(*args, **kwargs)[source]

Bases: django.contrib.auth.models.User

Deposit client

collections

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.contrib.auth.models.UserManager object>
provider_url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

domain

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exception DoesNotExist

Bases: django.contrib.auth.models.DoesNotExist

exception MultipleObjectsReturned

Bases: django.contrib.auth.models.MultipleObjectsReturned

deposit_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

user_ptr

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

restaurant.place is a ForwardOneToOneDescriptor instance.

user_ptr_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class swh.deposit.models.Deposit(*args, **kwargs)[source]

Bases: django.db.models.base.Model

Deposit reception table

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

reception_date

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

complete_date

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

collection

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

external_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

client

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

swh_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

swh_id_context

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

swh_anchor_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

swh_anchor_id_context

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status_detail

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

parent

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

client_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

collection_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

deposit_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

depositrequest_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

get_next_by_reception_date(**morekwargs)
get_previous_by_reception_date(**morekwargs)
get_status_display(**morekwargs)
objects = <django.db.models.manager.Manager object>
parent_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class swh.deposit.models.DepositRequestType(*args, **kwargs)[source]

Bases: django.db.models.base.Model

Deposit request type made by clients (either archive or metadata)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

depositrequest_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

objects = <django.db.models.manager.Manager object>
swh.deposit.models.client_directory_path(instance, filename)[source]

Callable to upload archive in MEDIA_ROOT/user_<id>/<filename>

Parameters:
  • instance (DepositRequest) – DepositRequest concerned by the upload
  • filename (str) – Filename of the uploaded file
Returns:

A path to be prefixed by the MEDIA_ROOT to access physically to the file uploaded.
class swh.deposit.models.DepositRequest(*args, **kwargs)[source]

Bases: django.db.models.base.Model

Deposit request associated to one deposit.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

deposit

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

date

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

raw_metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

archive

The descriptor for the file attribute on the model instance. Returns a FieldFile when accessed so you can do stuff like:

>>> from myapp.models import MyModel
>>> instance = MyModel.objects.get(pk=1)
>>> instance.file.size

Assigns a file object on assignment so you can do:

>>> with open('/path/to/hello.world', 'r') as f:
...     instance.file = File(f)
type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

deposit_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_date(**morekwargs)
get_previous_by_date(**morekwargs)
objects = <django.db.models.manager.Manager object>
type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class swh.deposit.models.DepositCollection(id, name)[source]

Bases: django.db.models.base.Model

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

deposit_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

objects = <django.db.models.manager.Manager object>

swh.deposit.parsers module

Module in charge of defining parsers with SWORD 2.0 supported mediatypes.

class swh.deposit.parsers.SWHFileUploadZipParser[source]

Bases: rest_framework.parsers.FileUploadParser

File upload parser limited to zip archive.

media_type = 'application/zip'
class swh.deposit.parsers.SWHFileUploadTarParser[source]

Bases: rest_framework.parsers.FileUploadParser

File upload parser limited to tarball (tar, tar.gz, tar.*) archives.

media_type = 'application/x-tar'
class swh.deposit.parsers.SWHXMLParser[source]

Bases: rest_framework.parsers.BaseParser

XML parser.

media_type = 'application/xml'
parse(stream, media_type=None, parser_context=None)[source]

Parses the incoming bytestream as XML and returns the resulting data.

class swh.deposit.parsers.SWHAtomEntryParser[source]

Bases: swh.deposit.parsers.SWHXMLParser

Atom entry parser limited to specific mediatype

media_type = 'application/atom+xml;type=entry'
parse(stream, media_type=None, parser_context=None)[source]

Parses the incoming bytestream as XML and returns the resulting data.

class swh.deposit.parsers.SWHMultiPartParser[source]

Bases: rest_framework.parsers.MultiPartParser

Multipart parser limited to a subset of mediatypes.

media_type = 'multipart/*; *'
swh.deposit.parsers.parse_xml(raw_content)[source]

Parse xml body.

Parameters:raw_content (bytes) – The content to parse
Returns:content parsed as dict.

swh.deposit.signals module

Module in charge of defining some uncoupled actions on deposit.

Typically, checking that the archives deposited are ok are not directly testing in the request/answer to avoid too long computations.

So this is done in the deposit_on_status_ready_for_check callback.

swh.deposit.signals.post_deposit_save(sender, instance, created, raw, using, update_fields, **kwargs)[source]
When a deposit is saved, check for the deposit’s status change and

schedule actions accordingly.

When the status passes to deposited, schedule checks. When the status pass to ready, schedule loading. Otherwise, do nothing.

Parameters:
  • sender (Deposit) – The model class
  • instance (Deposit) – The actual instance being saved
  • created (bool) – True if a new record was created
  • raw (bool) – True if the model is saved exactly as presented (i.e. when loading a fixture). One should not query/modify other records in the database as the database might not be in a consistent state yet
  • using – The database alias being used
  • update_fields – The set of fields to update as passed to Model.save(), or None if update_fields wasn’t passed to save()

swh.deposit.urls module

swhdeposit URL Configuration

swh.deposit.urls.default_view(req)[source]

swh.deposit.utils module

swh.deposit.utils.merge(*dicts)[source]

Given an iterator of dicts, merge them losing no information.

Parameters:*dicts – arguments are all supposed to be dict to merge into one
Returns:dict merged without losing information

Module contents