Package Bio :: Package SeqIO
[hide private]
[frames] | no frames]

Package SeqIO

source code

Sequence input/output as SeqRecord objects.

Bio.SeqIO is also documented at http://biopython.org/wiki/SeqIO and by a whole chapter in our tutorial:

Input

The main function is Bio.SeqIO.parse(...) which takes an input file handle, and format string. This returns an iterator giving SeqRecord objects:

>>> from Bio import SeqIO
>>> handle = open("Fasta/f002", "rU")
>>> for record in SeqIO.parse(handle, "fasta"):
...     print record.id, len(record)
gi|1348912|gb|G26680|G26680 633
gi|1348917|gb|G26685|G26685 413
gi|1592936|gb|G29385|G29385 471
>>> handle.close()

Note that the parse() function will all invoke the relevant parser for the format with its default settings. You may want more control, in which case you need to create a format specific sequence iterator directly.

Input - Single Records

If you expect your file to contain one-and-only-one record, then we provide the following 'helper' function which will return a single SeqRecord, or raise an exception if there are no records or more than one record:

>>> from Bio import SeqIO
>>> handle = open("Fasta/f001", "rU")
>>> record = SeqIO.read(handle, "fasta")
>>> handle.close()
>>> print record.id, len(record)
gi|3318709|pdb|1A91| 79

This style is useful when you expect a single record only (and would consider multiple records an error). For example, when dealing with GenBank files for bacterial genomes or chromosomes, there is normally only a single record. Alternatively, use this with a handle when download a single record from the internet.

However, if you just want the first record from a file containing multiple record, use the iterator's next() method:

>>> from Bio import SeqIO
>>> handle = open("Fasta/f002", "rU")
>>> record = SeqIO.parse(handle, "fasta").next()
>>> handle.close()
>>> print record.id, len(record)
gi|1348912|gb|G26680|G26680 633

The above code will work as long as the file contains at least one record. Note that if there is more than one record, the remaining records will be silently ignored.

Input - Multiple Records

For non-interlaced files (e.g. Fasta, GenBank, EMBL) with multiple records using a sequence iterator can save you a lot of memory (RAM). There is less benefit for interlaced file formats (e.g. most multiple alignment file formats). However, an iterator only lets you access the records one by one.

If you want random access to the records by number, turn this into a list:

>>> from Bio import SeqIO
>>> handle = open("Fasta/f002", "rU")
>>> records = list(SeqIO.parse(handle, "fasta"))
>>> handle.close()
>>> print records[1].id
gi|1348917|gb|G26685|G26685

If you want random access to the records by a key such as the record id, turn the iterator into a dictionary:

>>> from Bio import SeqIO
>>> handle = open("Fasta/f002", "rU")
>>> record_dict = SeqIO.to_dict(SeqIO.parse(handle, "fasta"))
>>> handle.close()
>>> len(record_dict)
3
>>> print len(record_dict["gi|1348917|gb|G26685|G26685"])
413

However, using list() or the to_dict() function will load all the records into memory at once, and therefore is not possible on very large files. Instead, for *some* file formats Bio.SeqIO provides an indexing approach providing dictionary like access to any record. For example,

>>> from Bio import SeqIO
>>> record_dict = SeqIO.index("Fasta/f002", "fasta")
>>> len(record_dict)
3
>>> print len(record_dict["gi|1348917|gb|G26685|G26685"])
413

Many but not all of the supported input file formats can be indexed like this. For example "ace", "embl", "fasta", "fastq", "genbank", "ig", "phd", "pir", "tab" and "qual" work, but alignment formats like "phylip", "clustalw" and "nexus" will not.

Input - Alignments

You can read in alignment files as Alignment objects using Bio.AlignIO. Alternatively, reading in an alignment file format via Bio.SeqIO will give you a SeqRecord for each row of each alignment:

>>> from Bio import SeqIO
>>> handle = open("Clustalw/hedgehog.aln", "rU")
>>> for record in SeqIO.parse(handle, "clustal"):
...     print record.id, len(record)
gi|167877390|gb|EDS40773.1| 447
gi|167234445|ref|NP_001107837. 447
gi|74100009|gb|AAZ99217.1| 447
gi|13990994|dbj|BAA33523.2| 447
gi|56122354|gb|AAV74328.1| 447
>>> handle.close()

Output

Use the function Bio.SeqIO.write(...), which takes a complete set of SeqRecord objects (either as a list, or an iterator), an output file handle and of course the file format:

   from Bio import SeqIO
   records = ...
   handle = open("example.faa", "w")
   SeqIO.write(records, handle, "fasta")
   handle.close()

In general, you are expected to call this function once (with all your records) and then close the file handle.

Output - Advanced

The effect of calling write() multiple times on a single file will vary depending on the file format, and is best avoided unless you have a strong reason to do so.

Trying this for certain alignment formats (e.g. phylip, clustal, stockholm) would have the effect of concatenating several multiple sequence alignments together. Such files are created by the PHYLIP suite of programs for bootstrap analysis.

For sequential files formats (e.g. fasta, genbank) each "record block" holds a single sequence. For these files it would probably be safe to call write() multiple times.

Conversion

The Bio.SeqIO.convert(...) function allows an easy interface for simple file format conversions. Additionally, it may use file format specific optimisations so this should be the fastest way too.

In general however, you can combine the Bio.SeqIO.parse(...) function with the Bio.SeqIO.write(...) function for sequence file conversion. Using generator expressions provides a memory efficient way to perform filtering or other extra operations as part of the process.

File Formats

When specifying the file format, use lowercase strings. The same format names are also used in Bio.AlignIO and include the following:

Note that while Bio.SeqIO can read all the above file formats, it cannot write to all of them.

You can also use any file format supported by Bio.AlignIO, such as "nexus", "phlip" and "stockholm", which gives you access to the individual sequences making up each alignment as SeqRecords.

Submodules [hide private]

Functions [hide private]
 
write(sequences, handle, format)
Write complete set of sequences to a file.
source code
 
parse(handle, format, alphabet=None)
Turns a sequence file into an iterator returning SeqRecords.
source code
 
_iterate_via_AlignIO(handle, format, alphabet)
Iterate over all records in several alignments (PRIVATE).
source code
 
_force_alphabet(record_iterator, alphabet)
Iterate over records, over-riding the alphabet (PRIVATE).
source code
 
read(handle, format, alphabet=None)
Turns a sequence file into a single SeqRecord.
source code
 
to_dict(sequences, key_function=None)
Turns a sequence iterator or list into a dictionary.
source code
 
index(filename, format, alphabet=None, key_function=None)
Indexes a sequence file and returns a dictionary like object.
source code
 
to_alignment(sequences, alphabet=None, strict=True)
Returns a multiple sequence alignment (OBSOLETE).
source code
 
convert(in_file, in_format, out_file, out_format, alphabet=None)
Convert between two sequence file formats, return number of records.
source code
 
_test()
Run the Bio.SeqIO module's doctests.
source code
Variables [hide private]
  _FormatToIterator = {"fasta": FastaIO.FastaIterator, "gb": Ins...
  _FormatToWriter = {"fasta": FastaIO.FastaWriter, "gb": InsdcIO...
  __package__ = 'Bio.SeqIO'
Function Details [hide private]

write(sequences, handle, format)

source code 

Write complete set of sequences to a file.

  • sequences - A list (or iterator) of SeqRecord objects.
  • handle - File handle object to write to.
  • format - lower case string describing the file format to write.

You should close the handle after calling this function.

Returns the number of records written (as an integer).

parse(handle, format, alphabet=None)

source code 

Turns a sequence file into an iterator returning SeqRecords.

  • handle - handle to the file.
  • format - lower case string describing the file format.
  • alphabet - optional Alphabet object, useful when the sequence type cannot be automatically inferred from the file itself (e.g. format="fasta" or "tab")

Typical usage, opening a file to read in, and looping over the record(s):

>>> from Bio import SeqIO
>>> filename = "Fasta/sweetpea.nu"
>>> for record in SeqIO.parse(open(filename,"rU"), "fasta"):
...    print "ID", record.id
...    print "Sequence length", len(record)
...    print "Sequence alphabet", record.seq.alphabet
ID gi|3176602|gb|U78617.1|LOU78617
Sequence length 309
Sequence alphabet SingleLetterAlphabet()

For file formats like FASTA where the alphabet cannot be determined, it may be useful to specify the alphabet explicitly:

>>> from Bio import SeqIO
>>> from Bio.Alphabet import generic_dna
>>> filename = "Fasta/sweetpea.nu"
>>> for record in SeqIO.parse(open(filename,"rU"), "fasta", generic_dna):
...    print "ID", record.id
...    print "Sequence length", len(record)
...    print "Sequence alphabet", record.seq.alphabet
ID gi|3176602|gb|U78617.1|LOU78617
Sequence length 309
Sequence alphabet DNAAlphabet()

If you have a string 'data' containing the file contents, you must first turn this into a handle in order to parse it:

>>> data = ">Alpha\nACCGGATGTA\n>Beta\nAGGCTCGGTTA\n"
>>> from Bio import SeqIO
>>> from StringIO import StringIO
>>> for record in SeqIO.parse(StringIO(data), "fasta"):
...     print record.id, record.seq
Alpha ACCGGATGTA
Beta AGGCTCGGTTA

Use the Bio.SeqIO.read(handle, format) function when you expect a single record only.

read(handle, format, alphabet=None)

source code 

Turns a sequence file into a single SeqRecord.

  • handle - handle to the file.
  • format - string describing the file format.
  • alphabet - optional Alphabet object, useful when the sequence type cannot be automatically inferred from the file itself (e.g. format="fasta" or "tab")

This function is for use parsing sequence files containing exactly one record. For example, reading a GenBank file:

>>> from Bio import SeqIO
>>> record = SeqIO.read(open("GenBank/arab1.gb", "rU"), "genbank")
>>> print "ID", record.id
ID AC007323.5
>>> print "Sequence length", len(record)
Sequence length 86436
>>> print "Sequence alphabet", record.seq.alphabet
Sequence alphabet IUPACAmbiguousDNA()

If the handle contains no records, or more than one record, an exception is raised. For example:

>>> from Bio import SeqIO
>>> record = SeqIO.read(open("GenBank/cor6_6.gb", "rU"), "genbank")
Traceback (most recent call last):
    ...
ValueError: More than one record found in handle

If however you want the first record from a file containing multiple records this function would raise an exception (as shown in the example above). Instead use:

>>> from Bio import SeqIO
>>> record = SeqIO.parse(open("GenBank/cor6_6.gb", "rU"), "genbank").next()
>>> print "First record's ID", record.id
First record's ID X55053.1

Use the Bio.SeqIO.parse(handle, format) function if you want to read multiple records from the handle.

to_dict(sequences, key_function=None)

source code 

Turns a sequence iterator or list into a dictionary.

  • sequences - An iterator that returns SeqRecord objects, or simply a list of SeqRecord objects.
  • key_function - Optional callback function which when given a SeqRecord should return a unique key for the dictionary.

e.g. key_function = lambda rec : rec.name or, key_function = lambda rec : rec.description.split()[0]

If key_function is ommitted then record.id is used, on the assumption that the records objects returned are SeqRecords with a unique id.

If there are duplicate keys, an error is raised.

Example usage, defaulting to using the record.id as key:

>>> from Bio import SeqIO
>>> handle = open("GenBank/cor6_6.gb", "rU")
>>> format = "genbank"
>>> id_dict = SeqIO.to_dict(SeqIO.parse(handle, format))
>>> print sorted(id_dict.keys())
['AF297471.1', 'AJ237582.1', 'L31939.1', 'M81224.1', 'X55053.1', 'X62281.1']
>>> print id_dict["L31939.1"].description
Brassica rapa (clone bif72) kin mRNA, complete cds.

A more complex example, using the key_function argument in order to use a sequence checksum as the dictionary key:

>>> from Bio import SeqIO
>>> from Bio.SeqUtils.CheckSum import seguid
>>> handle = open("GenBank/cor6_6.gb", "rU")
>>> format = "genbank"
>>> seguid_dict = SeqIO.to_dict(SeqIO.parse(handle, format),
...               key_function = lambda rec : seguid(rec.seq))
>>> for key, record in sorted(seguid_dict.iteritems()):
...     print key, record.id
/wQvmrl87QWcm9llO4/efg23Vgg AJ237582.1
BUg6YxXSKWEcFFH0L08JzaLGhQs L31939.1
SabZaA4V2eLE9/2Fm5FnyYy07J4 X55053.1
TtWsXo45S3ZclIBy4X/WJc39+CY M81224.1
l7gjJFE6W/S1jJn5+1ASrUKW/FA X62281.1
uVEYeAQSV5EDQOnFoeMmVea+Oow AF297471.1

This approach is not suitable for very large sets of sequences, as all the SeqRecord objects are held in memory. Instead, consider using the Bio.SeqIO.index() function (if it supports your particular file format).

index(filename, format, alphabet=None, key_function=None)

source code 

Indexes a sequence file and returns a dictionary like object.

  • filename - string giving name of file to be indexed
  • format - lower case string describing the file format
  • alphabet - optional Alphabet object, useful when the sequence type cannot be automatically inferred from the file itself (e.g. format="fasta" or "tab")
  • key_function - Optional callback function which when given a SeqRecord identifier string should return a unique key for the dictionary.

This indexing function will return a dictionary like object, giving the SeqRecord objects as values:

>>> from Bio import SeqIO
>>> records = SeqIO.index("Quality/example.fastq", "fastq")
>>> len(records)
3
>>> sorted(records.keys())
['EAS54_6_R1_2_1_413_324', 'EAS54_6_R1_2_1_443_348', 'EAS54_6_R1_2_1_540_792']
>>> print records["EAS54_6_R1_2_1_540_792"].format("fasta")
>EAS54_6_R1_2_1_540_792
TTGGCAGGCCAAGGCCGATGGATCA
<BLANKLINE>
>>> "EAS54_6_R1_2_1_540_792" in records
True
>>> print records.get("Missing", None)
None

Note that this psuedo dictionary will not support all the methods of a true Python dictionary, for example values() is not defined since this would require loading all of the records into memory at once.

When you call the index function, it will scan through the file, noting the location of each record. When you access a particular record via the dictionary methods, the code will jump to the appropriate part of the file and then parse that section into a SeqRecord.

Note that not all the input formats supported by Bio.SeqIO can be used with this index function. It is designed to work only with sequential file formats (e.g. "fasta", "gb", "fastq") and is not suitable for any interlaced file format (e.g. alignment formats such as "clustal").

For small files, it may be more efficient to use an in memory Python dictionary, e.g.

>>> from Bio import SeqIO
>>> records = SeqIO.to_dict(SeqIO.parse(open("Quality/example.fastq"), "fastq"))
>>> len(records)
3
>>> sorted(records.keys())
['EAS54_6_R1_2_1_413_324', 'EAS54_6_R1_2_1_443_348', 'EAS54_6_R1_2_1_540_792']
>>> print records["EAS54_6_R1_2_1_540_792"].format("fasta")
>EAS54_6_R1_2_1_540_792
TTGGCAGGCCAAGGCCGATGGATCA
<BLANKLINE>

As with the to_dict() function, by default the id string of each record is used as the key. You can specify a callback function to transform this (the record identifier string) into your prefered key. For example:

>>> from Bio import SeqIO
>>> def make_tuple(identifier):
...     parts = identifier.split("_")
...     return int(parts[-2]), int(parts[-1])
>>> records = SeqIO.index("Quality/example.fastq", "fastq",
...                       key_function=make_tuple)
>>> len(records)
3
>>> sorted(records.keys())
[(413, 324), (443, 348), (540, 792)]
>>> print records[(540, 792)].format("fasta")
>EAS54_6_R1_2_1_540_792
TTGGCAGGCCAAGGCCGATGGATCA
<BLANKLINE>
>>> (540, 792) in records
True
>>> "EAS54_6_R1_2_1_540_792" in records
False
>>> print records.get("Missing", None)
None

Another common use case would be indexing an NCBI style FASTA file, where you might want to extract the GI number from the FASTA identifer to use as the dictionary key.

Notice that unlike the to_dict() function, here the key_function does not get given the full SeqRecord to use to generate the key. Doing so would impose a severe performance penalty as it would require the file to be completely parsed while building the index. Right now this is usually avoided.

to_alignment(sequences, alphabet=None, strict=True)

source code 

Returns a multiple sequence alignment (OBSOLETE).

  • sequences -An iterator that returns SeqRecord objects, or simply a list of SeqRecord objects. All the record sequences must be the same length.
  • alphabet - Optional alphabet. Stongly recommended.
  • strict - Optional, defaults to True. Should error checking be done?

Using this function is now discouraged. Rather doing this:

>>> from Bio import SeqIO
>>> handle = open("Clustalw/protein.aln")
>>> alignment = SeqIO.to_alignment(SeqIO.parse(handle, "clustal"))
>>> handle.close()

You are now encouraged to use Bio.AlignIO instead, e.g.

>>> from Bio import AlignIO
>>> handle = open("Clustalw/protein.aln")
>>> alignment = AlignIO.read(handle, "clustal")
>>> handle.close()

convert(in_file, in_format, out_file, out_format, alphabet=None)

source code 

Convert between two sequence file formats, return number of records.

  • in_file - an input handle or filename
  • in_format - input file format, lower case string
  • out_file - an output handle or filename
  • out_format - output file format, lower case string
  • alphabet - optional alphabet to assume

NOTE - If you provide an output filename, it will be opened which will overwrite any existing file without warning. This may happen if even the conversion is aborted (e.g. an invalid out_format name is given).

For example, going from a filename to a handle:

>>> from Bio import SeqIO
>>> from StringIO import StringIO
>>> handle = StringIO("")
>>> SeqIO.convert("Quality/example.fastq", "fastq", handle, "fasta")
3
>>> print handle.getvalue()
>EAS54_6_R1_2_1_413_324
CCCTTCTTGTCTTCAGCGTTTCTCC
>EAS54_6_R1_2_1_540_792
TTGGCAGGCCAAGGCCGATGGATCA
>EAS54_6_R1_2_1_443_348
GTTGCTTCTGGCGTGGGTGGGGGGG
<BLANKLINE>

_test()

source code 

Run the Bio.SeqIO module's doctests.

This will try and locate the unit tests directory, and run the doctests from there in order that the relative paths used in the examples work.


Variables Details [hide private]

_FormatToIterator

Value:
{"fasta": FastaIO.FastaIterator, "gb": InsdcIO.GenBankIterator, "genba\
nk": InsdcIO.GenBankIterator, "genbank-cds": InsdcIO.GenBankCdsFeature\
Iterator, "embl": InsdcIO.EmblIterator, "embl-cds": InsdcIO.EmblCdsFea\
tureIterator, "ig": IgIO.IgIterator, "swiss": SwissIO.SwissIterator, "\
phd": PhdIO.PhdIterator, "ace": AceIO.AceIterator, "tab": TabIO.TabIte\
rator, "pir": PirIO.PirIterator, "fastq": QualityIO.FastqPhredIterator\
, "fastq-sanger": QualityIO.FastqPhredIterator, "fastq-solexa": Qualit\
yIO.FastqSolexaIterator, "fastq-illumina": QualityIO.FastqIlluminaIter\
...

_FormatToWriter

Value:
{"fasta": FastaIO.FastaWriter, "gb": InsdcIO.GenBankWriter, "genbank":\
 InsdcIO.GenBankWriter, "tab": TabIO.TabWriter, "fastq": QualityIO.Fas\
tqPhredWriter, "fastq-sanger": QualityIO.FastqPhredWriter, "fastq-sole\
xa": QualityIO.FastqSolexaWriter, "fastq-illumina": QualityIO.FastqIll\
uminaWriter, "phd": PhdIO.PhdWriter, "qual": QualityIO.QualPhredWriter\
,}