ATUC Meeting
Triennium Funding - Science Case

at the ATNF in Epping, Sydney

- March 23, 1999 -


Telescope Operators for the AT Compact Array

The Telescope Operator

The operator is the person responsible for the control and monitoring of the telescope. The ATCA is usually controlled via the computer program (user interface) CAOBS. The observation and Array systems are also monitored by computer - VIS, CAMON etc. Most important for monitoring is the program ASSISTANCE - a "well trained" program which can recognise a variety of system conditions which may threaten the data or the Array and in some cases offer the operator some limited advice on corrective actions. We can enumerate several types of telescope operator.

1. The Astronomer
At present (and since operations began in 1990) most observations are conducted by one of the astronomers associated with each project. The astronomer may be present at the Observatory, or operate the Array remotely via the computer network (usually from ATNF, Marsfield, but occasionally from other locations such as Hobart, Melbourne or Canberra). In the case of remote operation, the Duty Astronomer (DA) at the ATCA is responsible for assisting the astronomer in any operations not possible remotely, such as the manual setting or resetting of equipment.

2. Observatory staff
This option is discussed below.

3. None - automatic operation.
We might envisage routinely operating the Array unattended. In fact observations for individual projects run with little or no operator action required after the initial setup and START command. We should ask why we cannot, for example, run automatically from 4pm until 8am each day. In practice observations are often interrupted by events which require manual corrective actions but usually contribute very little to down-time.

i) Power interruptions - (antenna drives are not on UPS) These are frequent - perhaps once/day during summer. A simple restart is usually sufficient for recovery.
ii) Computer communications. Brief interruptions often arise from some loss of synchrony in inter-computer communications. Sometimes recovery is simple. At other times the symptoms can be confusing and considerable experience may be needed to recover. Completely eliminating this may be difficult while we continue to use a non-real-time operating system such as VMS in the control system.
iii) Weather. At present the antennas are stowed automatically on detection of high wind. However a manual restart is required. It would be possible to implement an automatic resumption of operation on declining wind speed. The reliance of automatic detection of impending thunderstorms needs further discussion.
iv) Other transient system faults. There is often some interruption from an undiagnosed system fault which may occur repeatedly for some time (days or more) until a solution is found. During that time recovery may be simple but requiring manual actions.

Clearly some of these interruptions could be handled automatically with modest changes to the current system. Perhaps such changes should be considered independent of the discussion on staff operators.

Staff operators.

1. How many?
Assuming operators work for 48 weeks each of 37 and we require operation all year (8766 hours) 5 (4.84) positions would be required.

2. Duties.

  • operate Array for observations (who would prepare SCHED files?);
  • operate Array for reconfigurations and calibration;
  • data management: acquisition; copies to archive and export; ship export data;
  • general performance monitoring;
  • if queue scheduling, select projects from pool;
  • other background tasks: write utility software; maintain documentation; maintain/extend ATCA calibrator list;
  • (possibly) preprocess all data before export.;
  • one of the operators would coordinate or direct the others.

    3. Budget
    i) Costs:

  • 5 salaries (including loadings) CSOF3M - $326k/annum.
  • Two additional cars to the Observatory fleet - $4k/annum.
  • If having operators led to no visiting astronomers, a decision to close the Lodge might be taken. In that case ATNF visitors would stay in a Narrabri motel. A rough estimate of cost is $20k/annum ($130/day (room + meals) for 150 nights). There would be other costs associated with closing the Lodge, such as to provide staff amenities. An alternative is to keep the Lodge open but at a much reduced level.

    ii) Savings (to ATNF)

  • travel - DA and ATNF astronomer trips between Sydney and Narrabri - $50k/annum
  • Lodge running costs $99k/annum
  • OR reduced Lodge costs - say halve the staffing level and save $43k/annum

    Arguments for and against

    For

    1. More efficient scheduling. With a staff operator it would be possible to use the queue scheduling (see below) and gain some efficiencies. In effect this would reduce the amount of unallocated time in the schedule. At present much of the time initially scheduled as unallocated is used for essential repair work or astronomy (DAs and observatory staff extending TAC approved projects, Target-of-Opportunity observations, restoring fault-induced time lost in scheduled observations). Perhaps an additional 500-1000 hours/annum could be used for astronomy. Some projects may gain additional benefit from not needing to be scheduled in the contiguous blocks used at present.
    2. Less down-time. Currently downtime is less than 5% of total scheduled time. Thus this saving could not exceed 300 hours/year.
    3. Operators may provide an efficient and consistent way to calibrate astronomers' data.
    4. No travel for astronomers, DAs. Some astronomers may argue that their time can be managed more efficently without the interruption and time spent in visiting the Observatory.

    Against

    1. Cost . The estimates above indicate that there would be a financial cost of about $250k/annum.
    2. Loss of astronomers visiting - direct feedback on Array performance, goals of Observatory, intellectual stimulation, other intangibles
    3. More black-box approach to Array - fewer people to offer informed suggestions on Array use and extensions
    4. Students - At present we contribute significantly to student training.

    Modes of Operation

    There are several ways to schedule the Array. A brief description of these is given as background.

    1. Fixed schedule
    Since operations began in 1990 the CA has been operated with a schedule which is prepared once each observing term of three or four months. Once prepared, only occasional changes are made, mainly

    2. "Queued" schedule
    Operating in this mode, a pool of ranked proposals would be formed (perhaps once per observing term). On some much shorter time scale, daily for example, the projects for the next time interval, one day in this case, would be selected according to rank, sidereal time, resources (eg. which receivers are available) and conditions (state of the atmosphere, position of the sun etc.).

    3. Mixed
    A hybrid mode could be envisaged where a scheduled is fixed for a whole observing term apart from some period within the term left open for a queued schedule. For example, a period of several weeks may be set aside for mm-wave observing whenever the atmosphere permits with cm-wave projects being chosen at other times.

    Dave McConnell
    22 February 1999


  • Meetings
    Public