The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package — TEI-tagged view

The Astropy CollaborationpersName, A. M. Price-WhelanpersName, B. M. SipőczpersName, H. M. GüntherpersName, P. L. LimpersName, S. M. CrawfordpersName, S. ConseilpersName, D. L. ShupepersName, M. W. CraigpersName, N. DenchevapersName, A. GinsburgpersName, J. T. VanderPlaspersName, L. D. BradleypersName, D. Pérez-SuárezpersName, M. de Val-BorropersName, (Primary Paper Contributors)persName, T. L. AldcroftpersName, K. L. CruzpersName, T. P. RobitaillepersName, E. J. TollerudpersName, (Astropy Coordination Committee)persName, C. ArdeleanpersName, T. BabejpersName, Y. P. BachpersName, M. BachettipersName, A. V. BakanovpersName, S. P. BamfordpersName, G. BarentsenpersName, P. BarmbypersName, A. BaumbachpersName, K. L. BerrypersName, F. BiscanipersName, M. BoquienpersName, K. A. BostroempersName, L. G. BoumapersName, G. B. BrammerpersName, E. M. BraypersName, H. BreytenbachpersName, H. BuddelmeijerpersName, D. J. BurkepersName, G. CalderonepersName, J. L. Cano RodríguezpersName, M. CarapersName, J. V. M. CardosopersName, S. CheedellapersName, Y. CopinpersName, L. CorralespersName, D. CrichtonpersName, D. D’AvellapersName, C. DeilpersName, É. DepagnepersName, J. P. DietrichpersName, A. DonathpersName, M. DroettboompersName, N. EarlpersName, T. ErbenpersName, S. FabbropersName, L. A. FerreirapersName, T. FinethypersName, R. T. FoxpersName, L. H. GarrisonpersName, S. L. J. GibbonspersName, D. A. GoldsteinpersName, R. GommerspersName, J. P. GrecopersName, P. GreenfieldpersName, A. M. GroenerpersName, F. GrollierpersName, A. HagenpersName, P. HirstpersName, D. HomeierpersName, A. J. HortonpersName, G. HosseinzadehpersName, L. HupersName, J. S. HunkelerpersName, Ž. IvezićpersName, A. JainpersName, T. JennesspersName, G. KanarekpersName, S. KendrewpersName, N. S. KernpersName, W. E. KerzendorfpersName, A. KhvalkopersName, J. KingpersName, D. KirkbypersName, A. M. KulkarnipersName, A. KumarpersName, A. LeepersName, D. LenzpersName, S. P. LittlefairpersName, Z. MapersName, D. M. MacleodpersName, M. MastropietropersName, C. McCullypersName, S. MontagnacpersName, B. M. MorrispersName, M. MuellerpersName, S. J. MumfordpersName, D. MunapersName, N. A. MurphypersName, S. NelsonpersName, G. H. NguyenpersName, J. P. NinanpersName, M. NöthepersName, S. OgazpersName, S. OhpersName, J. K. ParejkopersName, N. ParleypersName, S. PascualpersName, R. PatilpersName, A. A. PatilpersName, A. L. PlunkettpersName, J. X. ProchaskapersName, T. RastogipersName, V. Reddy JangapersName, J. SabaterpersName, P. SakurikarpersName, M. SeifertpersName, L. E. SherbertpersName, H. Sherwood-TaylorpersName, A. Y. ShihpersName, J. SickpersName, M. T. SilbigerpersName, S. SinganamallapersName, L. P. SingerpersName, P. H. SladenpersName, K. A. SooleypersName, S. SornarajahpersName, O. StreicherpersName, P. TeubenpersName, S. W. ThomaspersName, G. R. TremblaypersName, J. E. H. TurnerpersName, V. TerrónpersName, M. H. van KerkwijkpersName, A. de la VegapersName, L. L. WatkinspersName, B. A. WeaverpersName, J. B. WhitmorepersName, J. WoillezpersName, V. ZabalzapersName, (Astropy Contributors)persName (2018date). “The Astropy Project: Building an Open-science Project and Status of the v2.0 Core Package.” The Astronomical Journaltitle·j.

Object of study — the Astropy Projectterm's organization, infrastructure, and version-2.0 core package

Research question — how an open-development astronomy-software community organizes, sustains, and extends a shared Python core and interoperable package ecosystem

Contribution — a structured account of Astropyterm governance and infrastructure plus a technical status survey of the version-2.0 core package

Full text — tagged

The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.

The Astropy Projectterm: Building an Open-science Project and Status of the v2.0 Core Package Abstract The Astropy Projectterm supports and fosters the development of open-source and openly developed Python packages that provide commonly needed functionality to the astronomical community. A key element of the Astropy Projectterm is the core package astropyterm, which serves as the foundation for more specialized projects and packages. In this article, we provide an overview of the organization of the Astropy projectterm and summarize key features in the core package, as of the recent major releaseterm, version 2.0. We then describe the project infrastructure designed to facilitate and support development for a broader ecosystem of interoperable packages. We conclude with a future outlook of planned new features and directions for the broader Astropy Projectterm.

1. Introduction All modern astronomical research makes use of software in some way. Astronomy, as a field, has thus long supported the development of software tools for astronomical tasks, such as scripts that enable individual scientific research, software packages for small collaborationspersName, and data reduction pipelines for survey operations. Some software packages are, or were, supported by large institutions and intended for a wide range of users. These packages therefore typically provide some level of documentation and user support or training. Other packages are developed by individual researchers or research groups, are then typically used by smaller groups for more domain-specific purposes, and may have less extensive user support. For both packages meant for wider distribution and for scripts specific to particular research projects, a library that addresses common astronomical tasks simplifies the software development process by encouraging the reuse of common functions. The users of such a library then also benefit from a community and ecosystem built around a shared foundation. The Astropy projectterm has grown to become such a community for Python astronomy software, and the astropy core packageterm has become this shared foundation.

The development of the astropy core packageterm began as a largely community-driven effort to standardize core functionality for astronomical software in Python. In this way, its genesis differs from, but builds upon, many substantial and former astronomical software development efforts that were commissioned or initiated through large institutional support, such as IRAF (developed at NOAO; Tody 1993), MIDAS (developed at ESO; Banse et al. 1988), or Starlink (originally developed by a consortium of UK institutions and now maintained by the East Asian Observatory; Disney & Wallace 1982; Currie et al. 2014). More recently, community-driven efforts have seen significant success in the astronomical sciences (e.g., Turk et al. 2011).

Python is an increasingly popular, general-purpose programming language that is available under a permissive open-source software license and is free of charge for all major operating systems. This programming language has become especially popular in the quantitative sciences, where researchers must simultaneously conduct research, perform data analysis, and develop software. A large part of this success owes itself to the vibrant community of developers and a continuously growing ecosystem of tools, web servicesterm, and stable, well-developed packages that enable easier collaborationpersName on software development, easier writing and sharing of software documentation, and continuous testing and validation of software. While dedicated libraries provide support for array representation and arithmetic (numpyterm; Van der Walt et al. 2011), a wide variety of functions for scientific computing (scipy; Jones et al. 2001), and publication-quality plotting (matplotlib; Hunter 2007), tens of thousands of other high-quality and easy-to-use packages are available to help with tasks that are not specific to astronomy but might be performed in the course of astronomical research, e.g., interfacing with databases, or statistical inferences. More recently, the development and mainstream adoption of package managers such as Anaconda has significantly streamlined the installation process for many libraries, lowering the barriers to entry for new users.

The Astropy Projectterm aims to provide an open-source and open-development core package (astropyterm) and an ecosystem of affiliated packagesterm that support astronomical functionality in the Python programming language. “Open developmentterm” describes a process where anybody with an internet connection can suggest changes to the source code and contribute their opinion when new features, bug fixes or other code changes, governance, or any other aspect of the development process is discussed (see Sections 2.2 and 2.3 of how this is organized in practice). The astropy core packageterm is now a feature-rich library of sufficiently general tools and classes that supports the development of more specialized code. An example of such functionality is reading and writing FITS files: it would be time-consuming and impractical for multiple groups to implement the FITS standard (Pence et al. 2010) and maintain software for such a general-purpose need. Another example of such a common task is in dealing with representations of and transformations between astronomical coordinate systems.

The Astropy Projectterm aims to develop and provide high-quality code and documentation according to the best practices in software development. The project makes use of different tools and web servicesterm to reach those goals without central institutional oversight. The first public release of the astropy packageterm is described in Astropyterm CollaborationpersName et al. (2013). Since then, the astropy packageterm has been used in hundreds of projects and the scope of the package has grown considerably. At the same time, the scientific community contributing to the project has grown tremendously and an ecosystem of packages supporting or affiliated with the astropyterm core has developed. In this paper, we describe the current status of the Astropyterm community and the astropy core packageterm and discuss goals for future development.

We start by describing the way the Astropy Projectterm functions and is organized in Section 2. We then describe the main software efforts developed by the Astropy Projectterm itself: a core package called astropyterm (Section 3) and several separate packages that help maintain the infrastructure for testing and documentation (Section 4). We end with a short vision for the future of Astropyterm and astronomical software in general in Section 5. The full paper, including the code to produce the figures, is available in a GitHub repository (Price-WhelanpersName et al. 2018).

This article is not intended as an introduction to astropyterm, nor does it replace the astropyterm documentation. Instead, it describes the way the Astropyterm community is organized and the current state of the astropy packageterm.

2. Organization and Infrastructure 2.1. Coordination of Astropyterm From its inception, Astropyterm has required coordination to ensure the project as a whole and its coding efforts are consistent and reasonably efficient. While many Python projects adopt a “Benevolent Dictator For Life” (BDFL) modelterm, Astropyterm has instead opted for a coordination committee. This is partly due to the nature of the project as a large-scale collaborationpersName between many contributors with many interests, and partly due to the sheer amount of work that needs to get done. For the latter reason, the project has expanded the committee from three to four members starting in 2016.

For resolving disagreements about the astropy core packageterm or other Astropy-managed code, the coordination committee primarily acts to work toward consensus, or when consensus is difficult to achieve, generally acts as a “tie-breaker.” The committee also oversees affiliated packageterm applications to ensure that they are in keeping with Astropyterm’s vision and philosophy, as well as the associated procedures. Additionally, the committee oversees the assignment of roles (primarily driven by already-existing contributions), and increasingly has acted as the “face” of the Project, providing contact with organizations like NumFOCUS (the body that holds any potential funding in trust for Astropyterm) and the American Astronomical Society (AAS).

2.2. Astropyterm Development Modelterm Code is contributed to the astropy core packageterm or modified through “pull requeststerm” (via GitHub ) that often contain several git commits. Pull requeststerm may fix bugs, implement new features, or improve or modify the infrastructure that supports the development and maintenance of the package. Individual pull requeststerm are generally limited to a single conceptual addition or modification, to make code reviewterm tractable. Pull requeststerm that modify or add code to a specific subpackage must be reviewed and approved by one of the subpackage maintainers before they are merged into the core codebase. Bugs and feature requests are reported via the GitHub issue tracker and labeled with a set of possible labels that help classify and organize the issues. The development workflow is detailed in the astropyterm documentation.

As of version 2.0, astropyterm contains 212,244 lines of code contributed by 232 unique contributors over 19,270 git commits. Figure 1, left, shows the distribution of total number of commitsterm·var per contributor as of early 2018. The relative flatness of this distribution (as demonstrated by its log-log slope of −0.5) shows that the astropy core packageterm has been developed by a broad contributor base. A leading group of six developers have each added over 1000 commits to the repository, and ∼20 more core contributors have contributed at least 100 commits. However, the distribution of contribution level (number of commitsterm·var) continues from 100 down to a single commit. In this sense, the development of the core package has been a true community effort and is not dominated by a single individual. It is also important to note that the number of commitsterm·var is only a rough metric of contribution, as any single commit could be a critical fix in the package or merely a fix for a typographical error. Figure 1, middle, shows the number of commitsterm·var as a function of time since the genesis of the astropy core packageterm. The package is still healthy: new commits are and have been contributed at a steady rate throughout its existence. Figure 1, right, shows that, ≈20–25 people contribute to the core package each month. While we would like for this number to grow, this demonstrates that the core package is still being developed and maintained by a substantial group of contributors.

Figure 1. Left panel: distribution of number of commitsterm·var per committer. Middle panel: cumulative number of commitsterm·var to the astropy core packageterm over time. Right panel: number of unique committers per month to the astropy core packageterm.

Download figure: 2.3. APEs—Astropyterm Proposals for Enhancement The Astropy projectterm has a formal mechanism to propose significant changes to the core package (e.g., re-writing the coordinates subpackage; TollerudpersName et al. 2014), to plan out major new features (e.g., a new file format; AldcroftpersName 2015), or to institute new organization-wide policies (e.g., adopting a code of conduct; CruzpersName et al. 2015). This mechanism is called “Astropy Proposal for Enhancementterm” (APE) and is modeled after the “Python Enhancement Proposals” (PEP) that guide the development of the Python programming language. In an APE, one or more authors describe in detail the proposed changes or additions, including a rationale for the changes, how these changes will be implemented, and in the case of code, what the interface will be (GreenfieldpersName 2013). The APEs are discussed and refined by the community before much work is invested into a detailed implementation; anyone is welcome to contribute to these discussions during the open consideration period. APEs are proposed via pull requeststerm on a dedicated GitHub repository ; therefore, anyone can read the proposed APEs and leave in-line comments. When a community consensus emerges, the APEs are accepted and become the basis for future work. In cases where consensus cannot be reached, the Astropyterm coordination committee may decide to close the discussion and make an executive decision based on the community input on the APE in question.

2.4. Concept of Affiliated Packagesterm A major part of the Astropy Projectterm is the concept of “Affiliated Packagesterm.” An affiliated packageterm is an astronomy-related Python package that is not part of the astropy core packageterm, but has requested to be included as part of the Astropy Project’s communityterm. These packages support the goals and vision of Astropyterm of improving code re-use, interoperability, and embracing good coding practices such as testing and thorough documentation.

Affiliated packagesterm contain functionality that is more specialized, have license incompatibilities, or have external dependencies (e.g., GUI libraries) that make these packages more suitable to be separate from the astropy core packageterm. Affiliated packagesterm may also be used to develop substantial new functionality that will eventually be incorporated into the astropy core packageterm (e.g., astropy.visualization.wcsaxesterm). New functionality benefits from having a rapid development and release cycle that is not tied to that of the astropyterm core (Section 2.5). These projects may also have less stringent requirements for style, testing, or development as compared to the core package.

Affiliated packagesterm are listed on the main Astropyterm website and advertised to the community through Astropyterm mailing lists; a list of current affiliated packagesterm is included in Table 1. Becoming an affiliated packageterm is a good way for new and existing packages to gain exposureterm while promoting Astropyterm’s high standard for code and documentation quality. This process of listing and promoting affiliated packagesterm is one way in which the Astropy Projectterm tries to increase code re-use in the astronomical community.

Packages can become affiliated with Astropyterm by applying for this status on a public mailing list. The coordination committee (Section 2.1) reviews such requests and issues recommendations for the improvement of a package, where applicable.

2.5. Release Cycle and Long-term Supportterm The astropy packageterm has a regular release schedule consisting of new significant releases every six months, with bugfix releasesterm as needed (TollerudpersName 2013). The major releasesterm contain new features or any significant changes, whereas the bugfix releasesterm only contain fixes to code or documentation but no new features. Some versions are additionally designated as “Long-term supportterm” (LTS) releases, which continue to receive bug fixes for two years following the release with no changes to the API. The LTS versions are ideal for pipelines and other applications in which API stability is essential. The latest LTS release (version 2.0) is also the last one that supports Python 2; it will receive bug fixes until the end of 2019 (RobitaillepersName 2017).

The version numbering of the astropy core packageterm reflects this release scheme: the core package version number uses the form x.y.z, where “x” is advanced for LTS releases, “y” for non-LTS feature releases, and “z” for bugfix releasesterm. This is similar to Semantic Versioning.

The released versions of the astropy core packageterm are available from several of the Python distributions for scientific computing (e.g., Anaconda) and from the Python Package Index (PyPI). Effort has been made to make astropyterm available and easily installable across all platforms; the package is constantly tested on different platforms as part of a suite of continuous integration tests.

2.6. Support of Astropyterm The Astropy Projectterm, as of the version 2.0 release, does not receive any direct financial support for the development of astropyterm. Development of the software, all supporting materials, and community support are provided by individuals who work on the Astropy Projectterm in their own personal time, by individuals or groups contributing to Astropyterm as part of a research project, or contributions from institutions that allocate people to work on Astropyterm. A list of organizations that have contributed to Astropyterm in this manner can be found in the Acknowledgments.

Different funding modelsterm have been proposed for support of Astropyterm (e.g., MunapersName et al. 2016), but a long-term plan for sustainability has not yet been established. The Astropy Projectterm has the ability to accept financial contributions from institutions or individuals through the NumFOCUS organization. NumFOCUS has, to date, covered the direct costs incurred by the Astropy Projectterm.

2.7. Reuse of the Scientific Python Ecosystem The Astropy Projectterm is built on a philosophy of building from the existing Python scientific ecosystemterm wherever possible. Many of the enabling technologies like numpyterm, scipy, matplotlib, or cython, are necessary as the core underpinnings of Astropy packagesterm. Often, this means using these packages as “building blocks” (e.g., the ubiquitous use of numpyterm arrays throughout astropyterm). In other cases, this means wrappers around more general algorithms that make them more convenient for astronomy use cases (e.g., the model-fitting in the astropyterm.modeling subpackage discussed in Section 3.7). Sometimes these simple wrappers evolve into more complex implementations that address astronomically relevant use cases the general tool does not support (e.g., astropyterm.convolution, see Section 3.8). As a broad rule, the Project explicitly encourages re-use of code where possible.

However, the boundaries of when this re-use is called for is often ambiguous. Some of the examples above only evolved after significant debate in the community over whether these algorithms were sufficient. Other times, even apparently general functionality did not exist in the wider ecosystem that met the Astropyterm community’s needs, and hence the functionality had to be developed wholly from the developer resources available in the community (e.g., astropyterm.units, or astropyterm.table when it began). At the same time, however, the Astropy Projectterm has provided the wider community with myriad bug fixes to the enabling technologies listed above, as well as the testing and documentation architecture. Functionality is only “extracted” from Astropyterm to other packages after careful consideration that includes considering the impact on the maintenance and support of Astropyterm.

3. Astropy Core Packageterm Version 2.0 The Astropy Projectterm aims to provide Python-based packages for all tasks that are commonly needed in a large subset of the astronomical community. At the foundation is the astropy core packageterm, which provides general functionality (e.g., coordinate transformationsterm, reading and writing astronomical files, and units) or base classes for other packages to utilize for a common interface (e.g., NDData). In this section, we highlight new features introduced or substantially improved since version 0.2 (previously described in Astropyterm CollaborationpersName et al. 2013). The astropy packageterm provides a full log of changes over the course of the entire project and more details about individual subpackages are available in the documentation. Beyond what is mentioned below, most subpackages have seen improved performance since the release of the version 0.2 package.

3.1. Units The astropyterm.units subpackage adds support for representing units and numbers with associated units—“quantitiesterm·def”—in code. Historically, quantitiesterm·def in code have often been represented simply as numbers, with units implied or noted via comments in the code because of considerations about speed: having units associated with numbers inherently adds overhead to numerical operations. In astropyterm.units, Quantity objectsterm extend numpyterm array objects and have been designed with speed in mind.

As of astropyterm version 2.0, units and quantitiesterm·def, prevalent in most of its subpackages, have become a key concept for using the package as a whole. Units are intimately entwined in the definition of astronomical coordinates; thus, nearly all functionality in the astropyterm.coordinates subpackage (see Section 3.3) depends on them. Most astropyterm subpackages have been made compatible with Quantity objectsterm, although they are not always required.

The motivation and key concepts behind this subpackage were described in detail in the previous paper (Astropyterm CollaborationpersName et al. 2013). Therefore, we primarily highlight new features and improvements here.

3.1.1. Interaction with numpyterm Arrays Quantity objectsterm extend numpy.ndarray objectsterm and therefore work well with many of the functions in numpyterm that support array operations. For example, Quantity objectsterm with angular units can be directly passed in to the trigonometric functions implemented in numpyterm. The units are internally converted to radians, which is what the numpyterm trigonometric functions expect, before being passed to numpyterm.

3.1.2. Logarithmic Units and Magnitudes By default, taking the logarithm of a Quantity objectterm with non-dimensionless units intentionally fails. However, some well-known units are actually logarithmic quantitiesterm·def, where the logarithm of the value is taken with respect to some reference value. Examples include astronomical magnitudes, which are logarithmic fluxes, and decibels, which are more generic logarithmic ratios of quantitiesterm·def. Logarithmic, relative units are now supported in astropyterm.units.

3.1.3. Defining Functions that Require Quantitiesterm·def When writing code or functions that expect Quantity objectsterm, we often want to enforce that the input unitsterm have the correct physical type. For example, we may want to require only length-type Quantity objectsterm. astropyterm.units provides a tool called quantity_input()term that can perform this verification automatically to avoid repetitive code.

3.2. Constants The astropyterm.constants subpackage provides a selection of physical and astronomical constants as Quantity objectsterm (see Section 3.1). A brief description of this package was given in Astropyterm CollaborationpersName et al. (2013). In version 2.0, the built-in constants have been organized into modules for specific versions of the constant values. For example, physical constants have codata2014 (Mohr et al. 2016) and codata2010 versions. Astronomical constants are organized into iau2015 and iau2012 modules to indicate their sources (resolutions from the International Astronomical Union, IAU). The codata2014 and iau2015 versions are combined into the default constant value version: astropyconst20. For compatibility with astropyterm version 1.3, astropyconst13 is available and provides access to the adopted versions of the constants from earlier versions of astropyterm. To use previous versions of the constants as units (e.g., solar masses), the values have to be imported directly; with version 2.0, astropyterm.units uses the astropyconst20 versions.

Astronomers using astropyterm.constants should take particular note of the constants provided for Earth, Jupiter, and the Sun. Following IAU 2015 Resolution B3 (Mamajek et al. 2015), nominal values are now given for mass parameters and radii. The nominal values will not change even as “current best estimates” are updated.

3.3. Coordinates The astropyterm.coordinates subpackage is designed to support representing and transforming celestial coordinates and—new in version 2.0—velocities. The framework heavily relies on the astropyterm.units subpackage, and most inputs to objects in this subpackage are expected to be Quantity objectsterm. Some of the machinery also relies on the Essential Routines of Fundamental Astronomy (ERFA) C library for some of the critical underlying transformation machinery (TollerudpersName et al. 2017), which is based on the Standards Of Fundamental Astronomy (SOFA) effort (Hohenkerk 2011).

A key concept behind the design of this subpackage is that coordinate representationsterm and reference systems/framesterm are independent of one another. For example, a set of coordinates in the International Celestial Reference System (ICRS) reference frame could be represented as spherical (right ascension, declination, and distance from solar system barycenter) or Cartesian coordinates (x, y, z with the origin at barycenter). They can therefore change representations independent of being transformed to other reference frames (e.g., the Galactic coordinate frame).

The classes that handle coordinate representationsterm (the Representation classes) act like three-dimensional vectors and thus support vector arithmetic. The classes that represent reference systems and frames (the Frame classesterm) internally use Representation objectsterm to store the coordinate data—that is, the Frame classesterm accept coordinate data, either as a specified Representation objectterm, or using short-hand keyword arguments to specify the components of the coordinates. These preferred representation and short-hand component names differ between various astronomical reference systems. For example, in the ICRS frame, the spherical angles are right ascension (ra) and declination (dec), whereas in the Galactic frame, the spherical angles are Galactic longitude (l) and latitude (b). Each Frame classterm defines its own component names and preferred Representation class. The frame-specific component names map to corresponding components on the underlying Representation objectterm that internally stores the coordinate data. For most frames, the preferred representation is spherical, although this is determined primarily by their common use in the astronomical community.

Many of the Frame classesterm also have attributes specific to the corresponding reference system that allow the user to specify the frame. For example, the Fifth Fundamental Catalogue (FK5) reference system requires specifying an equinox to determine the reference frame. If required, these additional frame attributes must be specified, along with the coordinate data, when a Frame object is created. Figure 2 shows the network of possible reference frame transformations as currently implemented in astropyterm.coordinates. Custom user-implemented Frame classesterm that define transformations to any reference frame in this graph can then be transformed to any of the other connected frames.

Figure 2. The full graph of possible reference frame transformations implemented in astropyterm.coordinates. Arrows indicate transformations from one frame to another. Arrows that point back to the same frame indicate self-transformations that involve a change of reference frame parameters (e.g., equinox).

Download figure: The typical user does not usually have to interact directly with the Frame or Representation classes. Instead, astropyterm.coordinates provides a high-level interface to representing astronomical coordinates through the SkyCoord classterm, which was designed to provide a single class that accepts a wide range of possible inputs. It supports coordinate data in any coordinate frame in any representation by internally using the Frame and Representation classes.

In what follows, we briefly highlight key new features in astropyterm.coordinates.

3.3.1. Local Earth Coordinate Frames In addition to representing celestial coordinates, astropyterm now supports specifying positions on the Earth in a number of different geocentric systems with the EarthLocation classterm. With this, astropyterm now supports Earth-location-specific coordinate systems such as the altitude-azimuth (AltAzterm) or horizontal system. Transformations between AltAzterm and any Barycentric coordinate frame also requires specifying a time using the Time classterm from astropyterm.time. With this new functionality, many of the common tasks associated with observation planning can now be completed with astropyterm or the Astropy-affiliated package astroplan (MorrispersName et al. 2018).

3.3.2. Proper Motion and Velocity Transformations In addition to positional coordinate data, the Frame classesterm now also support velocity data. As the default representation for most frames is spherical, most of the Frame classesterm expect proper motion and radial velocity components to specify the velocity information. The names of the proper motion components all start with pm and adopt the same longitude and latitude names as the positional components. Transforming coordinates with velocity data is also supported, but in some cases the transformed velocity components have limited accuracy because the transformations are done numerically instead of analytically. The low-level interface for specifying and transforming velocity data is currently experimental. As such, in version 2.0, only the Frame classesterm (and not the SkyCoord classterm) support handling velocities.

3.3.3. Solar System Ephemerides Also new is support for computing ephemerides of major solar system bodies and outputting the resulting positions as coordinate objects. These ephemerides can be computed either using analytic approximations from ERFA or from downloaded JPL ephemerides (the latter requires the jplephem optional dependency and an internet connection).

3.3.4. Accuracy of Coordinate Transformationsterm In order to check the accuracy of the coordinate transformationsterm in astropyterm.coordinates, we have created a set of benchmarks that we use to compare transformations between a set of coordinate frames for a number of packages. Because no package can be guaranteed to implement all transformations to arbitrary precision and some transformations are sometimes subject to interpretation of standards (particularly in the case of Galactic coordinates), we do not designate any of the existing packages as the “ground truth” but instead compare each tool to all other tools. The benchmarks are thus useful beyond the Astropy Projectterm because they allow all of the tools to be compared to all other tools. The tools included in the benchmark at the moment include the astropy core packageterm, Kapteyn (Terlouw & Vogelaar 2015), NOVAS (Barron et al. 2011), PALpy (JennesspersName & BerrypersName 2013), PyAST (a wrapper for AST, described in BerrypersName et al. 2016), PyTPM, PyEphem (Rhodes 2011), and pySLALIB (a Python wrapper for SLALIB, described in Wallace 1994).

The benchmarks are meant to evolve over time and include an increasing variety of cases. At the moment, the benchmarks are set up as follows—we have generated a standard set of 1000 pairs of random longitudes/latitudesterm that we use in all benchmarks. Each benchmark is then defined using an input and output coordinate frameterm·var, using all combinations of FK4, FK5, Galactic, ICRS, and Ecliptic frames. For now, we set the epoch of observation to J2000. We also set the frame to J2000 (for FK5 and Ecliptic) and B1950 (for FK4). In the future, we plan to include a larger variety of epochs and equinoxes, as well as tests of conversion to/from Altitude/Azimuth. For each benchmark, we convert the 1000 longitudes/latitudes from the input/output frame with all tools and quantify the comparison by looking at the median, mean, maximum, and standard deviationterm·var of the absolute separation of the output coordinates from each pair of tools.

Figure 3 visualizes the relative accuracy of the conversion from FK4 to Galactic coordinates for all pairs of tools that implement this transformation. In this figure, the color of the cell indicates the maximum difference (in arcseconds) between the two tools over the 1000 longitude-latitude pairs tested. This figure shows, for example, that astropyterm, Kapteyn, and PyTPM agree with sub-milliarcsecond differences (light colors, small differences), while PALpy, pySLALIB, and PyAST also agree among themselves. However, there is an offset of around 02 between the two groups. Finally, PyEphem disagrees with all other packages by 04–08 (darker colors, large differences). These values are only meant to be illustrative and will change over time as the benchmarks are refined and the packages updated.

Figure 3. Comparison matrix of the maximum difference between longitude-latitude values in a set of 1000 random points transformed from FK4 to Galactic with the different packages. Darker colors (larger differences) are more significant disagreements.

Download figure: 3.4. Time The astropyterm.time subpackage focuses on supporting timescales (e.g., UTC, TAI, UT1) and time formats (e.g., Julian date, modified Julian date) that are commonly used in astronomy. This functionality is needed, for example, to calculate barycentric corrections or sidereal times. astropyterm.time is currently built on the ERFA (TollerudpersName et al. 2017) C library, which replicates the Standards of Fundamental Astronomy (SOFA; Hohenkerk 2011) but is licensed under a three-clause BSD license. The package was described in detail in Astropyterm CollaborationpersName et al. (2013) and has remained stable for the last several versions of astropyterm. Thus, in what follows, we only highlight significant changes or new features since the previous Astropyterm paper.

3.4.1. Barycentric and Heliocentric Corrections Detailed eclipse or transit timing requires accounting for light traveltime differences from the source to the observatory because of the Earth’s motion. It is therefore common to instead convert times to the solar system barycenter or heliocenter where the relative timing of photons is standardized. With the location of a source on the sky (i.e., a SkyCoord object), the location of an observatory on Earth (i.e., an EarthLocation object), and time values as Time objects, the time corrections to shift to the solar system barycenter or heliocenter can now be computed with astropyterm.time using the light_travel_time method of a Time object.

3.5. Data Containers 3.5.1. nddata The astropyterm.nddata subpackage provides three types of functionality: an abstract interface for representing generic arbitrary-dimensional data sets intended primarily for subclassing by developers of other packages, concrete classes building on this interface, and utilities for manipulating these kind of data sets.

The NDDataBase class provides the abstract interface for gridded data with attributes for accessing metadata, the world coordinate system (WCS), uncertainty arrays matched to the shape of the data, and other traits. Building on this interface, the NDData class provides a minimal working implementation for storing numpyterm arrays. These classes serve as useful base classes for package authors wishing to develop their own classes for specific use cases and as containers for exchanging gridded data.

The classes NDDataRef, NDDataArray, and CCDData extend the base storage functionality with options to do basic arithmetic (addition, subtraction, multiplication, and division), including error propagation in limited cases, and slicing of the data set based on grid coordinates that appropriately handles masking, errors, and units (if present). Additionally, the CCDData class also provides reading and writing from and to FITS files, and uses data structures from astropyterm, like WCS, to represent the file contents abstractly.

The http://docs.astropyterm.org/en/stable/nddata/utils.html astropyterm.nddata.utils module provides utilities that can operate on either plain numpyterm arrays or any of the classes in the astropyterm.nddata subpackage. It features a class for representing two-dimensional image cutouts, allowing one to easily link pixels in the cutout to those in the original image or vice versa, to convert between world and pixel coordinates in the cutout, and to overlay the cutout on images. Functions to enlarge or reduce an image by doing block replication or reduction are also provided.

3.5.2. Tables The astropyterm.table subpackage provides functionality for representing and manipulating heterogeneous data. In some respects, this is similar to numpyterm record arrays (Van der Walt et al. 2011) or pandas DataFrame objects (McKinney 2010), but with modifications for astronomical data. Most notably, tables from astropyterm.table allow for table or column metadata and can handle vectors/arrays or arbitrary objects (with suitable column-like characteristics) as table entries. The subpackage was described in detail in Astropyterm CollaborationpersName et al. (2013). Thus, in what follows, we only summarize key new features or updates to astropyterm.table since the previous Astropyterm paper. These are support for grouped table operations, table concatenation, and using array-valued astropyterm objects as table columns.

A table can contain data that naturally form groups; for example, it may contain multiple observations of a few sources at different points in time and in different bands. We may want to split the table into groups based on the combination of source observed and the band, after which we combine the results for each combination of source and band in some way (e.g., finding the mean or standard deviation of the fluxes or magnitudes over time) or filter the groups based on user-defined criteria. These kinds of grouping and aggregation operations are now fully supported by Table objects.

Table objects can now be combined in several different ways. If two tables have the same columns, we may want to stack them “vertically” to create a new table with the same columns but all rows. If two tables are row-matched but have distinct columns, we may want to stack them “horizontally” to create a new table with the same rows but all columns. For other situations, more generic table concatenation or joining are also possible when two tables share some columns.

The Table object now allows array-valued Quantity, celestial coordinate (SkyCoord), and date/time (Time) objects to be used as columns. It also provides a general way for other user-defined array-like objects to be used as columns. This makes it possible, for instance, to easily represent catalogs of sources or time series in Astropyterm, while having both the benefits of the Table object (e.g., accessing specific rows/columns or groups of them and combining tables) as well as, for example, the SkyCoord or the Time classesterm (e.g., converting the coordinates to a different frame or accessing the date/time in the desired timescale).

Additionally, there is now interoperability between Table and pandas DataFrame objects using the Table.to_pandas() and Table.from_pandas() methods. Table.to_pandas() is, however, limited to a subset of possible Table objects, because several of the features outlined above cannot be represented as pandas data frames. While these features might be contributed to or combined with pandas at a later date, the architectural differences that enable these areas are significant, so this would be a major undertaking that would require substantial investment from the core maintainers and communities of both packages.

3.6. io The astropyterm.io subpackages provide support for reading and writing data to a variety of ASCII and binary file formats, such as a wide range of ASCII data table formats, FITS, HDF5, and VOTable. It also provides a unified interface for reading and writing data with these different formats using the astropyterm.table subpackage. For many common cases, this simplifies the process of file input and output (I/O) and reduces the need to master the separate details of all the I/O packages within astropyterm. The file interface allows transparent compression of the gzip, bzip2, and lzma (.xz) formats; the latter two require the Python installation to have been compiled with support the respective libraries.

3.6.1. ASCII One of the problems when storing a table in an ASCII format is preserving table metadata such as comments, keywords and column data types, units, and descriptions. The newly defined Enhanced Character Separated Values (ECSVterm, AldcroftpersName 2015) format makes it possible to write a table to an ASCII-format file and read it back with no loss of information. The ECSVterm format has been designed to be both human-readable and compatible with most simple CSV readers.

The astropyterm.io.ascii subpackage now includes a significantly faster Cython/C engineterm for reading and writing ASCII files. This is available for most of the common formats. It also offers some additional features like parsing of different exponential notation styles, such as commonly produced by Fortran programs. On average, the new engine is about four to five times faster than the corresponding pure-Python implementationterm and is often comparable to the speed of the pandas (McKinney 2010) ASCII file interface. The fast reader has a parallel processing option that allows harnessing multiple cores for input parsing to achieve even greater speed gains. By default, read() and write() will attempt to use the fast Cython/C engineterm when dealing with compatible formats. Certain features of the full read / write interface are unavailable in the fast version, in which case the reader will by default fall back automatically to the pure-Python version.

The astropyterm.io.ascii subpackage now provides the capability to read a table within an HTML file or web URL into an astropyterm Table object. A Table object can now also be written out as an HTML table.

3.6.2. FITS The astropyterm.io.fits subpackage started as a direct port of the PyFITS project (Barrett & Bridgman 1999). Therefore, it is pretty stable, with mostly bug fixes but also a few new features and performance improvements. The API remains mostly compatible with PyFITS, which is now deprecated in favor of astropyterm.

Command-line scripts are now available for printing a summary of the HDUs in FITS file(s) (fitsinfo) and for printing the header information to the screen in a human-readable format (fitsheader).

FITS files are now loaded lazily by default, i.e., an object representing the list of HDUs is created but the data are not loaded into memory until requested. This approach should provide substantial speed-ups when using the convenience functions (e.g., getheader() or getdata()) to get an HDU that is near the beginning in a file with many HDUs.

3.6.3. HDF5 The astropyterm.io.misc.hdf5 subpackage provides support for binary read and write access to files in the Hierarchical Data Format (HDF5), if the h5py package is installed. Astropyterm table I/O is offered transparently through Table.read() and Table.write(), analogously to the other auto-detected formats. The keyword path=’group/subgroup/data set’ specifies the path to the data inside the file’s hierarchical structure.

3.7. Modeling The astropyterm.modeling subpackage provides a framework for representing analytical modelsterm and performing modelterm evaluation and parameter fitting. The main motivation for this functionality was to create a framework that allows arbitrary combination of modelsterm to support the Generalized World Coordinate System (GWCS) package. The current FITS WCS specification lacks the flexibility to represent arbitrary distortions and does not meet the needs of many types of current instrumentation. The fact that the astropyterm modeling framework now supports propagating units also makes it a useful tool for representing and fitting astrophysical modelsterm within data analysis tools.

Modelsterm and fittersterm are independent of each other: a modelterm can be fit with different fittersterm and new fittersterm can be added without changing existing modelsterm. The framework is designed to be flexible and easily extensible. The goal is to have a rich set of modelsterm, but also facilitate creating new ones, if necessary.

3.7.1. Single Modelterm Definition and Evaluation Modelsterm are defined by their parameters and initialized by providing values for them. The names of the parameters are stored in a list, Modelterm.param_names. Parameters are complex objects. They store additional information—default value, default unit, and parameter constraints. Parameter valuesterm·var and constraints can be updated by assignment. Supported constraints include fixed and tied parameters, as well as bounds on parameter valuesterm·var. The framework also supports modelsterm for which the number of parameters and their names are defined by another argument. A typical example is a polynomial modelterm defined by its degree. A modelterm is evaluated by calling it as a function.

If an analytical inverse of a modelterm exists, it can be accessed by calling Modelterm.inverse. In addition, Modelterm.inverse can be assigned another modelterm that represents a computed inverse.

Another useful settable property of modelsterm is Model.bounding_boxterm. This attribute sets the domain over which the modelterm is defined. This greatly improves the efficiency of evaluationterm when the input range is much larger than the characteristic width of the modelterm itself.

3.7.2. Modelterm Sets Using astropyterm.modeling provides an efficient way to set up the same type of modelterm with many different sets of parameter valuesterm·var. This creates a modelterm set that can be efficiently evaluated. For example, in PSF (point-spread function) photometry, all objects in an image will have a PSF of the same functional form, but with different positions and amplitudes.

3.7.3. Compound Modelsterm Modelsterm can be combined using arithmetic expressions. The result is also a modelterm, which can further be combined with other modelsterm. Modeling supports arithmetic (+, −, *, /, and **), join (&), and composition (∣) operators. The rules for combining modelsterm involve matching their inputs and outputs. For example, the composition operator, ∣, requires the number of outputs of the left modelterm to be equal to the number of inputs of the right one. For the join operator, the total number of inputs must equal the sum of number of inputs of both the left and the right modelsterm. For all arithmetic operators, the left and the right modelsterm must have the same number of inputs and outputs. An example of a compound modelterm could be a spectrum with interstellar absorption. The stellar spectrum and the interstellar extinction are represented by separate modelsterm, but the observed spectrum is fitted with a compound modelterm that combines both.

3.7.4. Fitting Modelsterm to Data The astropyterm.modeling subpackage also provides several fittersterm that are wrappers around some of the numpyterm and scipy.optimize functions and provide support for specifying parameter constraints. The fittersterm take a modelterm and data as input and return a copy of the modelterm with the optimized parameter valuesterm·var set. The goal is to make it easy to extend the fitting framework to create new fittersterm. The optimizers available in astropyterm version 2.0 are Levenberg–Marquardt (scipy.optimize.leastsq), Simplex (scipy.optimize.fmin), SLSQP (scipy.optimize.slsqp), and LinearLSQFitter (numpyterm.linalg.lstsq which provides exact solutions for linear modelsterm).

Modeling also supports a plugin system for fittersterm, which allows using the astropyterm modelsterm with external fittersterm. An example of this is SABA, which is a bridge between Sherpa (Doe et al. 2007), and astropyterm.modeling, to bring the Sherpa fittersterm into astropyterm.

3.7.5. Creating New Modelsterm If arithmetic combinations of existing modelsterm are not sufficient, new modelterm classes can be defined in different ways. The astropyterm.modeling package provides tools to turn a simple function into a full-featured modelterm, but it also allows extending the built-in modelterm class with arbitrary code.

3.7.6. Unit Support The astropyterm.modeling subpackage now supports the representation, evaluation, and fitting of modelsterm using Quantity objectsterm, which attach units to scalar values or arrays of values. In practice, this means that one can, for example, fit a modelterm to data with units and get parameters that also have units out, or initialize a modelterm with parameters with units and evaluate it using input values with different but equivalent units. For example, the blackbody modelterm (BlackBody1D) can be used to fit observed flux densities in a variety of units and as a function of different units of spectral coordinates (e.g., wavelength or frequency).

3.8. Convolution The astropyterm.convolution subpackage implements normalized convolutionterm (e.g., Knutsson & Westin 1993), an image reconstruction technique in which missing dataterm are ignored during the convolution and replaced with values interpolated using the kernel. An example is given in Figure 4. In astropyterm versions ≤1.3, the direct convolution and Fast Fourier Transform (FFT) convolution approaches were inconsistent, with only the latter implementing normalized convolutionterm. As of version 2.0, the two methods now agree and include a suite of consistency checks.

Figure 4. An example showing different modes of convolution available in the Python ecosystem. Each red x signifies a pixel that is set to NaN in the original data (top left). If the data are convolved with a Gaussian kernel on a 9 × 9 grid using scipy’s direct convolution (top right), any pixel within range of the original NaN pixels is also set to NaN. The bottom left panel shows what happens if the NaNs are set to zero first: the original NaN regions are depressed relative to their surroundings. Finally, the bottom right panel shows astropyterm’s convolution behavior, where the missing pixels are replaced with values interpolated from their surroundings using the convolution kernel.

Download figure: 3.9. Visualization The astropy.visualization subpackageterm provides functionality that can be helpful when visualizing data. This includes a framework (previously the standalone astropy.visualization.wcsaxesterm package) for plotting astronomical images with coordinates via matplotlib, functionality related to image normalization (including both scaling and stretching), smart histogram plotting, red-green-blue (RGB) color image creation from separate images, and custom plotting styles for matplotlib.

3.9.1. Image Stretching and Normalization Using astropyterm.visualization provides a framework for transforming values in images (and more generally, any arrays), typically for the purpose of visualization. The two main types of transformations are normalization and stretching of image values.

Normalization transforms the image values to the range [0, 1] using lower and upper limitsterm·var (vmin, vmax), where x represents the values in the original image.

Stretching transforms the image values in the range [0, 1] again to the range [0, 1], using a linear or nonlinear function: Several classes are provided for automatically determining intervals (e.g., using image percentiles) and for normalizing values in this interval to the [0, 1] range.

Using matplotlib allows a custom normalization and stretch to be used when displaying data by passing in a normalization object. The astropyterm.visualization package also provides a normalization class that wraps the interval and stretches objects into a normalization object that matplotlib understands.

3.9.2. Plotting Image Data with World Coordinates Astronomers dealing with observational imaging commonly need to make figures with images that include the correct coordinates and optionally display a coordinate grid. The challenge, however, is that the conceptual coordinate axes (such as longitude/latitude) need not be lined up with the pixel axes of the image. The astropy.visualization.wcsaxesterm subpackage implements a generalized way of making figures from an image array and a WCS object that provides the transformation between pixel and world coordinates.

World coordinates can be, for example, right ascension and declination, but can also include, for example, velocity, wavelength, frequency, or time. The main features from this subpackage include the ability to control which axes to show which coordinate on (e.g., showing longitude ticks on the top and bottom axes and latitude on the left and right axes), controlling the spacing of the ticks either by specifying the positions to use or providing a tick spacing or an average number of ticks that should be present on each axis, setting the format for the tick labels to ones commonly used by astronomers, controlling the visibility of the grid/graticule, and overlaying ticks, labels, and/or grid lines from different coordinate systems. In addition, it is possible to pass data with more than two dimensions and slice on-the-fly. Last but not least, it is also able to define non-rectangular frames such as, for example, Aitoff projections.

This subpackage differs from APLpyterm (RobitaillepersName & Bressert 2012), in that the latter focuses on providing a very high-level interface to plotting that requires very few lines of code to get a good result, whereas astropy.visualization.wcsaxesterm defines an interface that is much closer to that of matplotlib (Hunter 2007). This enables significantly more advanced visualizations.

An example of a visualization made with astropy.visualization.wcsaxesterm is shown in Figure 5. This example illustrates the ability to overlay multiple coordinate systems and customize which ticks/labels are shown on which axes around the image. This also uses the image stretching functionality from Section 3.9.1 to show the image in a square-root stretch (automatically updating the tick positions in the colorbar).

Figure 5. An example of a figure made using the astropy.visualization.wcsaxesterm subpackage, using Spitzer/IRAC 8.0 μm data from the Cygnus-X Spitzer Legacy survey (Beerer et al. 2010).

Download figure: 3.9.3. Choosing Histogram Bins The astropy.visualization subpackageterm also provides a histogram function, which is a generalization of matplotlib’s histogram function, to allow for a more flexible specification of histogram bins. The function provides several methods of automatically tuning the histogram bin sizeterm·var. It has a syntax identical to matplotlib's histogram function, with the exception of the bins parameter, which allows specification of one of four different methods for automatic bin selection: “blocks,” “knuth,” “scott,” or “freedman.”

3.9.4. Creating Color RGB Images Lupton et al. (2004) describe an “optimal” algorithm for producing RGB composite images from three separate high-dynamic range arrays. The astropy.visualization subpackageterm provides a convenience function to create such a color image. It also includes an associated set of classes to provide alternate scalings. This functionality was contributed by developers from the Large Synoptic Survey Telescope (LSST) and serves as an example of a contribution to Astropyterm from a more traditional engineering organization (JennesspersName et al. 2016).

The Sloan Digital Sky Survey (SDSS) SkyServer color images were made using a variation on this technique. As an example, in Figure 6, we show an RGB color image of the Hickson 88 group, centered near NGC 6977. This image was generated from SDSS images using the astropyterm.visualization tools.

Figure 6. An RGB color image of the region near the Hickson 88 group constructed from SDSS images and the astropyterm.visualization tools. This example uses astropy.visualization.wcsaxesterm to display the sky coordinate grid, and the http://docs.astropyterm.org/en/stable/api/astropyterm.visualization.make_lupton_rgb.html make_lupton_rgb() function to produce the RGB image from three SDSS filter images (g, r, i). The image on the left shows the image with the default parameters, whereas the image on the right has parameters set to show a greater dynamical range.

Download figure: 3.10. Cosmology The astropyterm.cosmology subpackage contains classes for representing different cosmologies and functions for calculating commonly used quantitiesterm·def such as look-back time and distance. The subpackage was described in detail in Astropyterm CollaborationpersName et al. (2013). The default cosmology in astropyterm version 2.0 is given by the values in Planck CollaborationpersName et al. (2016).

3.11. Statistics The astropy.stats packageterm provides statistical tools that are useful for astronomy and are either not found in or extend the available functionality of other Python statistics packages, such as scipy (Jones et al. 2001) or statsmodels (Seabold & Perktold 2010). The astropy.stats packageterm also contains a range of functionality used by many different disciplines in astronomy. It is not a complete set of statistical tools, but rather a still growing collection of useful features.

3.11.1. Robust Statistical Estimators Robust statisticsterm provide reliable estimates of basic statistics for complex distributions that largely mitigate the effects of outliersterm. The astropy.stats packageterm includes several robust statistical functionsterm that are commonly used in astronomy, such as sigma clipping methods for rejecting outliers, median absolute deviation functions, and biweight estimators, which have been used to calculate the velocity dispersion of galaxy clusters (Beers et al. 1990).

3.11.2. Circular Statistics Astronomers often need to compute statistics of quantitiesterm·def evaluated on a circle, such as sky direction or polarization angle. A set of circular statistical estimators based on Jammalamadaka & Sengupta (2001) are implemented in astropyterm.stats. These functions provide measurements of the circular mean, variance, and moment. All of these functions work with both numpyterm.ndarrays (assumed to be in radians) and Quantity objectsterm. In addition, the subpackage includes tests for Rayleigh Test, vtest, and a function to compute the maximum likelihood estimator for the parameters of the von Mises distribution.

3.11.3. Lomb–Scargle Periodograms Periodic analysis of unevenly spaced time series is common across many subfields of astronomy. The astropy.stats packageterm now includes several efficient implementations of the Lomb–Scargle periodogram (Lomb 1976; Scargle 1982) and several generalizations, including floating mean modelsterm (Zechmeister & Kürster 2009), truncated Fourier modelsterm (Bretthorst 2003), and appropriate handling of heteroscedastic uncertainties. Importantly, the implementations make use of several fast and scalable computational approaches (e.g., Press & Rybicki 1989; Palmer 2009), and thus can be applied to much larger data sets than Lomb–Scargle algorithms available in, e.g., scipy.stats (Jones et al. 2001). Much of the Lomb–Scargle code in astropyterm has been adapted from previously published open-source code (VanderPlaspersName et al. 2012; VanderPlaspersName & Ivezic 2015). Users should be aware that correct interpretation of periodogram results involves some subtleties; for a thorough discussion of this issue, see (VanderPlaspersName 2018, in press).

3.11.4. Bayesian Blocksterm and Histogram Binning The astropy.stats packageterm also includes an implementation of Bayesian Blocksterm (Scargle et al. 2013), an algorithm for analysis of breakpoints in nonperiodic astronomical time-series. One interesting application of Bayesian Blocksterm is its use in determining optimal histogram binnings, particularly binnings with unequal bin sizes. This code was adapted, with several improvements, from the astroML package (VanderPlaspersName et al. 2012). An example of a histogram fit using the Bayesian Blocksterm algorithm is shown in the right panel of Figure 7.

Figure 7. Three approaches to a 1D histogram. Left: a standard histogram using matplotlib’s default of 10 bins. Center: a histogram with the number of equal-width bins determined automatically using numpyterm’s bins = ’auto’. Right: a histogram created by astropyterm, with irregularly spaced bins computed via the Bayesian Blocksterm algorithm. Compared to regularly spaced bins, the irregular bin widths give a more accurate visual representation of features in the data set at various scales.

Download figure: 4. Infrastructure for Astropyterm Affiliated Packagesterm In addition to astronomy-specific packages and libraries, the Astropy Projectterm also maintains and distributes several general-purpose infrastructure packages that assist with the maintenance and upkeep of the astropy core packageterm and other affiliated packagesterm. The following sections describe the most widely used infrastructure packages developed by the Astropy Projectterm.

4.1. Package Template Astropyterm provides a package template—as a separate GitHub repository, astropyterm/package-template —that aims to simplify setting up packaging, testing, and documentation builds for developers of affiliated packagesterm or astropy-dependent packages. Any Python package can make use of this ready-to-go package layout, setup, installation, and Sphinx documentation build infrastructure that was originally developed for the astropy core packageterm and affiliated packagesterm maintained by the Astropy Projectterm. The package template also provides a testing framework, template configurations for continuous integration services, and Cython build support.

4.2. Continuous Integration Helpers Astropyterm also provides a set of scripts for setting up and configuring continuous integration (CI) services as a GitHub repository, astropyterm/ci-helpers. These tools aim to enable package maintainers to control their testing setup and installation process for various CI services through a set of environment variables. While the current development is mostly driven by the needs of the Astropyterm ecosystem, the actual usage of this package is extremely widespread. The current tools support configuration for Travis CI and Appveyor CI.

4.3. Sphinx Extensions The documentation for many Python packages, including all the packages in the Astropyterm ecosystem, is written using the Sphinx documentation build system. Sphinx supports writing documentation using plain text files that follow a markup language called reStructuredText (RST). These files are then transformed into HTML, PDF, or LATEX documents during the documentation build process. For the Astropy Projectterm, we have developed several Sphinx extensions that facilitate automatically generating API documentation for large projects, like the astropy core packageterm. The main extension we have developed is sphinx-automodapi, which provides a convenient single RST command to generate a set of documentation pages, listing all of the available classes, functions, and attributes in a given Python module.

5. The Future of the Astropy Projectterm Following the release of version 2.0, development on the next major version of the astropy core packageterm (version 3.0) began. On top of planned changes and additions to the core package, we also plan to overhaul the Astropyterm educational/learning materials and further generalize the infrastructure utilities originally developed for the core package for the benefit of the community.

5.1. Future Versions of the Astropyterm Core and Affiliated Packagesterm One of the most significant changes coming in this next major releaseterm will be removing the support for Python 2term (RobitaillepersName 2017): future versions of astropyterm will only support Python 3.5 or higher. Removing Python 2 support will allow the use of new features exclusive to Python 3, simplify the code base, and reduce the testing overheadterm for the package. Version 3.0 was released in February 2018.

In the next major releaseterm after version 3.0, scheduled for mid-2018, the focus will be on algorithm optimization and documentation improvement. To prepare for this release, we are subjecting the core package to testing, evaluation, and performance monitoring. As a result, less new functionality may be introduced, as a trade-off for better performance.

Beyond the core code, the Astropy Projectterm is also further developing the Astropy-managed affiliated packagesterm. While these may not be integrated into the astropy core packageterm, these projects provide code that is useful to the astronomical community and meet the testing and documentation standards of Astropyterm. Some of these new efforts include an initiative to develop tools for spectroscopy (CrawfordpersName et al. 2017, specutils, specreduc, specviz), integration of LSST software, and support for HEALPIX projection.

5.2. Learn Astropyterm The documentation of the astropy core packageterm contains narrative descriptions of the package’s functionality, along with detailed usage notes for functions, classes, and modules. While useful as a reference for more experienced Python users, it is not the proper point of entry for other users or learning environments. In the near future, we will launch a new resource for learning to use both the astropy core packageterm and the many packages in the broader Astropyterm ecosystem, under the name Learn Astropyterm.

The new Learn Astropyterm site will present several different ways to engage with the Astropyterm ecosystem: We encourage any users who wish to see specific material to either contribute or comment on these efforts via the Astropyterm mailing list or astropyterm/astropy-tutorials GitHub repository.

6. Conclusion The astropy packageterm is improving in stability, breadth, and reliability while the the Astropy projectterm is still significantly growing. As the astropy core packageterm becomes more mature, several subpackages have reached stability with a rich set of features that help astronomers worldwide to perform many daily tasks, such as planning observations, analyzing data or simulation results, and writing publications. The strong emphasis that the Astropy Projectterm puts on reliability and high coding standards helps users to trust the calculations performed with astropyterm and to publish reproducible results. At the same time, the Astropyterm ecosystem and core package are both growing: new functionality is still being contributed and new affiliated packagesterm are being developed to support more specialized needs.

The Astropy Projectterm is also spreading awareness of best practices in community-driven software development. This is important because most practicing astronomers were not explicitly taught computer science and software development, despite the fact that a substantial fraction of many astronomers’ workload today is related to software use and development. The astropy packageterm leads by example, showing all interested astronomers how modern tools like git version control or CI testing can increase the quality, accessibility, and discoverability of astronomical software without overly complicating the development cycle. Within Astropyterm, all submitted code is reviewed by at least one (but typically more than one) member of the Astropyterm community. Reviewers provide feedbackterm to contributors, which helps to improve contributors’ software development skillsterm. As a community, Astropyterm follows an explicit code of conduct (CruzpersName et al. 2015) and treats all contributors and users with respect, provides a harassment-free environment, and encourages and welcomes new contributions from all. Thus, while the Astropy Projectterm provides and develops software and tools essential to modern astronomical research, it also helps to prepare the current and next generation of researchers with the knowledge to adequately use, develop, and contribute to those tools within a conscientious and welcoming community.

We thank the referee for an insightful report that improved the paper. We would like to thank the members of the community who have contributed to Astropyterm, who have opened issues and provided feedbackterm, and who have supported the project in a number of different ways. We would like to acknowledge Alex Conley and Neil Crighton for maintaining the astropyterm.cosmology subpackage.

The Astropyterm community is supported by and makes use of a number of organizations and services outside the traditional academic community. We thank Google for financing and organizing the Google Summer of Code (GSoC) program, that has funded several students per year to work on Astropyterm related projects over the summer. These students often turn into long-term contributors. We also thank NumFOCUS and the Python Software Foundation for financial support. Within the academic community, we thank institutions that make it possible for astronomers and other developers on their staff to contribute their time to the development of Astropy projectsterm. We acknowledge the support of the Space Telescope Science Institute, Harvard–Smithsonian Center for Astrophysics, and the South African Astronomical Observatory.

The following individuals would like to recognize support for their personal contributions. H.M.G. was supported by the National Aeronautics and Space Administration through the Smithsonian Astrophysical Observatory contract SV3-73016 to MIT for Support of the Chandra X-Ray Center, which is operated by the Smithsonian Astrophysical Observatory for and on behalf of the National Aeronautics Space Administration under contract NAS8-03060. J.T.V. was supported by the UW eScience Institute via grants from the Moore Foundation, the Sloan Foundation, and the Washington Research Foundation. S.M.C. acknowledges the National Research Foundation of South Africa. M.V.B. was supported by NASA’s Planetary Astronomy Program. T.L.A. was supported by NASA contract NAS8-03060. Support for E.J.T. was provided by NASA through Hubble Fellowship grant No. 51316.01 awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., for NASA, under contract NAS 5-26555, as well as a Giacconi Fellowship. M.B. was supported by the FONDECYT regular project 1170618 and the MINEDUC-UA projects codes ANT 1655 and ANT 1656. D.H. was supported through the SFB 881 “The Milky Way System” by the German Research Foundation (DFG). W.E.K was supported by an ESO Fellowship. C.M. is supported by NSF grant AST-1313484. S.P. was supported by grant AYA2016-75808-R (FEDER) issued by the Spanish government. J.E.H.T. was supported by the Gemini Observatory, which is operated by the Association of Universities for Research in Astronomy, Inc., on behalf of the international Gemini partnership of Argentina, Brazil, Canada, Chile, and the United States of America. Y.P.B was supported by the Korea Astronomy and Space Science Institute, under the R&D program supervised by the Ministry of Science, ICT, and Future Planning.

Furthermore, the astropy packagesterm would not exist in their current form without a number of web servicesterm for code hosting, continuous integration, and documentation; in particular, astropyterm heavily relies on GitHub, Travis CI, Appveyor, CircleCI, and Read the Docs.

Astropyterm interfaces with the SIMBAD database, operated at CDS, Strasbourg, France. It also makes use of the ERFA library (TollerudpersName et al. 2017), which in turn derives from the IAU SOFA Collection developed by the International Astronomical Union Standards of Fundamental Astronomy (Hohenkerk 2011).

Software: astropyterm (Astropyterm CollaborationpersName et al. 2013), numpyterm (Van der Walt et al. 2011), scipy (Jones et al. 2001), matplotlib (Hunter 2007), Cython (Behnel et al. 2011), SOFA (Hohenkerk 2011), ERFA (TollerudpersName et al. 2017).

Appendix: List of Affiliated Packagesterm The Appendix comprises Table 1.

Table 1. Registry of Affiliated Packagesterm Package Name Stable PyPI Name Maintainer Citation APLpyterm Yes APLpyterm ThomaspersName RobitaillepersName and Eli Bressert RobitaillepersName & Bressert (2012) Astro-SCRAPPY Yes astroscrappy Curtis McCullypersName van Dokkum (2001) astroML Yes astroML Jake VanderplaspersName VanderplaspersName et al. (2012), IvezićpersName et al. (2014) astroplan astroplan Brett MorrispersName MorrispersName et al. (2018) astroquery Yes astroquery Adam GinsburgpersName and Brigitta Sipocz GinsburgpersName et al. (2017b) ccdproc Yes ccdproc Steven CrawfordpersName, Matt CraigpersName, and Michael SeifertpersName CraigpersName et al. (2015) cluster-lensing cluster-lensing Jes Ford Ford (2016) gala Yes astro-gala Adrian Price-WhelanpersName Price-WhelanpersName (2017) galpy Yes galpy Jo Bovy Bovy (2015) gammapy gammapy Christoph DeilpersName DeilpersName et al. (2017) ginga Yes ginga Eric Jeschke and Pey-Lian LimpersName ejeschke et al. (2017) Glue Yes glueviz Chris Beaumont and ThomaspersName RobitaillepersName Beaumont et al. (2014) gwcs gwcs Nadia DenchevapersName DenchevapersName et al. (2017) Halotools Yes halotools Andrew Hearin Hearin et al. (2017) HENDRICS Yes hendrics Matteo BachettipersName BachettipersName (2015) hips hips Christoph DeilpersName and ThomaspersName Boch hips developers (2018) imexam imexam Megan Sosey Sosey (2017) linetools Yes linetools J. Xavier ProchaskapersName, Nicolas Tejos, and Neil Crighton ProchaskapersName et al. (2017) marxs Yes marxs Hans Moritz GüntherpersName GüntherpersName et al. (2017) naima Yes naima Victor ZabalzapersName ZabalzapersName (2015) omnifit Yes omnifit Aleksi Suutarinen Suutarinen (2015) photutils photutils Larry BradleypersName and Brigitta Sipocz BradleypersName et al. (2017) poliastro poliastro Juan Luis Cano RodríguezpersName RodríguezpersName et al. (2017) PyDL pydl Benjamin Alan WeaverpersName WeaverpersName et al. (2017) pyregion Yes pyregion Jae-Joon LeepersName and Christoph DeilpersName pyregions developers (2018) pyspeckit Yes pyspeckit Adam GinsburgpersName GinsburgpersName & Mirocha (2011) python-cpl python-cpl Ole StreicherpersName StreicherpersName & Weilbacher (2012) PyVO pyvo Stefan Becker Graham et al. (2014) regions regions Christoph DeilpersName and Johannes KingpersName pyregions developers (2018) reproject Yes reproject ThomaspersName RobitaillepersName RobitaillepersName (2018) sncosmo Yes sncosmo Kyle Barbary Barbary (2014) spectral-cube Yes spectral-cube Adam GinsburgpersName GinsburgpersName et al. (2017a) specutils specutils Nicholas EarlpersName, Adam GinsburgpersName, Steve CrawfordpersName, and Erik TollerudpersName specutils developers (2018) spherical_geometry spherical-geometry Bernie Simon stingray stingray Daniela Huppenkothen, Matteo BachettipersName, Abigail Stevens, Simone Migliari, and Paul Balm Huppenkothen et al. (2016) synphot Yes synphot Pey Lian LimpersName LimpersName (2016) Download table as: ASCIITypeset image