FAQ: Experiment Installation & Maintenance​

EXPERIMENT INSTALLATION | PRE-PAMM | TUNNEL ACCESS | HARDWARE

 

EXPERIMENT INSTALLATION

Question:
What are the steps necessay to install my experiment in the tunnel?

Answer:
Assuming your experiment has been approved, Christine Clarke will organise the installation.

1. Get your apparatus to SLAC.

2. Stage your apparatus outside of FACET (we have space available in End Station B's clean room and we can also arrange space in MFD's *very* clean room) to confirm that all the neccessary parts are present and in good condition to be installed. Because of this stage, you need to have your parts delivered to SLAC well in advance. At least a day is necessary to stage and check all the items and then if there are any issues, it could take around a week to resolve.

There fore you must deliver your apparatus to SLAC 1-2 weeks before installation. Sooner is preferred.

3. Install it. Usually the experimenters are aided by Test Facilities staff or MFD Vacuum technicians depending on the type of work involved. Christine arranges the support. Usually this will occur during FACET downtime.

Sometimes we install apparatus on access days though this is not preferred and due to the difficulties in scheduling this, this can delay beam delivery to the experiment.

For a description of how to submit work in FACET during a downtime, go here.

For a description on how to submit work in FACET for an access (called "PAMM") day, go here.


Question:
​How do I write the procedure for experiment installation?

Answer:
Contact the FACET User Manager Christine Clarke and tell her what you would like to do at least a week in advance (but preferably even earlier) for efficient planning of the activity. The sooner the better in order that we can guarantee arrange appropriate support.

Christine needs to have a written procedure from you.
 
The procedure should include the following details:

  • Purpose/goal for the work
  • Number of people involved (give names if known)
  • Steps for the work (with time duration estimates per step and in total)
  • Hazards involved and safety measures that must be taken

The level of detail for the work should increase with its complexity and whether the job is "new" rather than routinely performed. Routine and simple jobs may only be briefly described.
 
Examples of good proceedures can be found here and here. These are written by two very different user groups in very different styles but both show good planning.
 
The main purpose of this procedure is for you, the user. It should be written in the format you find easiest to work to. Writing it should be a good exercise for you that ensures safe and successful tunnel work.
 
A generic safety analysis for experiments in FACET can be found here. This can be useful to consult when you plan your work though please do not copy and paste hazards/mitigations - only write the key points that actually pertain to your work and are useful in the execution of it. Complex experiments such as E200 have their own safety analyses (including beryllium windows, laser etc) and all workers should understand the hazards of their own hardware.

 

Question:
​What are the rules for in-vacuum hardware?

Answer:
Components need to be vacuum compatible to 1e-6 Torr (or mbar).

​General guidelines for bolts in vacuum chambers (written by Robert Ariniello):

  • If the bolt is threaded into a blind hole (anything that isn't a through hole) it needs to be vented. Otherwise a small pocket of gas can be trapped under the bolt and leak out slowly over time.
  • If the bolt is threaded into a stainless steel part, it should be coated. If it isn't coated galling can occur and the bolt can seize in the threads. This isn't strictly required for a good vacuum, but if bolts aren't coated they may be very difficult to remove. Silver plated is the most common coating but you can also use MOS2 or gold.
  • Bolts must be made of stainless steel and not have zinc coatings.

Silver plated bolts can be purchased from McMaster-Carr and UC Components, vented bolts can be purchased McMaster-Carr, UC Components, and Thorlabs, bolts that are both vented and silver plated come from UC Components. It took around 1 month to get bolts from UC Components in November when I last ordered from them.

Some of the holes on the top of a UTS stage are blind and need to be vented. The PB [Picnic Basket vacuum chamber] breadboard is stainless steel and needs to have silver bolts, but they don't need to be vented.

 


PRE-PAMM

Question:
What is a PAMM?

Answer:
A PAMM is a "Planned Access for Maintenance and Modifications". Typically during FACET running there was a PAMM every one to two weeks on Tuesday and Wednesday. This is subject to change for FACET-II as we find our feet.

The beam is shut off at a pre-arranged time. After a one hour radiation cooling off period, the tunnel is ready for access by Radiation Physics Field Operations (RPFO). RPFO survey the beamline. After the survey, RPFO de-briefs the access coordinator and any workers present for the start of the access.  

Only jobs that have beed pre-approved and released are allowed to be done.

If you keep the FACET User Manager aware of your access needs in general, we can coordinate the work.

 

Question:
How are the jobs for a PAMM requested and released?

Answer:
1). Requested work procedures are sent to the FACET User Manager (Christine Clarke). Usually this has to happen the Thursday before the PAMM.

2). Christine puts them into CATER (Comprehensive Accelerator Tool for Enhancing Reliability!).

3). Area Manager, Al Owens, reviews the CATER and releases the job (or not if he finds fault with the job).

4). Released and unreleased jobs are reviewed in a PAMM planning meeting on Monday at 1pm. Christine represents all experimenter jobs though attendance is open to all.

5) Released jobs are reviewed in a Radiation Physics meeting on Tuesday morning.

Workers are *required* to have authorisation from their supervisors prior to work. Non-employees ("users") are usually supervised by Christine who authorises the work by reviewing the procedure. If there is any doubt, Christine will contact the supervisors to check whether the worker is authorised to perform the requested job.

If the term "release" in this answer are unfamiliar to you, you shouldn't be requesting jobs in FACET! All these terms are explained in course 120 which is required for working in FACET. You can retake the course at any time as a refresher or read the ES&H chapter on Work Planning and Control.

 

Question:
How can I add an item to the list of jobs for a PAMM?

Answer:
As soon as you know you want to work in the tunnel, contact the FACET User Manager Christine Clarke and explain what you would like to do before COB on the Thursday before the upcoming access. Christine needs to know approximate duration and some details of the work especially location and number of people. This puts the work on the schedule. Christinewill let you know the date the work is scheduled.

On the Thursday before the access/scheduled work, send Christine the written procedure for the work.

The procedure should include the following details:

  • Purpose/goal for the work
  • Number of people involved (give names if known)
  • Steps for the work (with time duration estimates per step and in total)
  • Hazards involved and safety measures that must be taken

The level of detail for the work should increase with its complexity and whether the job is "new" rather than routinely performed. Routine and simple jobs may only be briefly described.
 
Examples of good proceedures can be found here and here. These are written by two very different user groups in very different styles but both show good planning.
 
The main purpose of this procedure is for you, the user. It should be written in the format you find easiest to work to. Writing it should be a good exercise for you that ensures safe and successful tunnel work.
 
A generic safety analysis for experiments in FACET can be found here. This can be useful to consult when you plan your work though please do not copy and paste hazards/mitigations - only write the key points that actually pertain to your work and are useful in the execution of it. Complex experiments such as E200 have their own safety analyses (including beryllium windows, laser etc) and all workers should understand the hazards of their own hardware.


User groups who routinely write procedures are more successful on their PAMM days. Experienced user groups can attest to this!

 

Question:
Can I add to the list of approved PAMM jobs after the Thursday deadline?

Answer:
​The PAMM job final deadline is 10:00AM on the Monday before the Tuesday or Wednesday PAMM. Only emergencies will be accepted after this point.

(All PAMM jobs need to be reviewed prior to release so if you miss this deadline without a good excuse, we are unlikely to be able to review it in time for it to proceed).
 

Question:
Why are you asking for the PAMM jobs so early? (I don't know what I want to do a week in advance!)

Answer:
The idea is to encourage planning ahead and prevent last minute additions.

The AOSD Maintenance Office politely ask people to have their jobs in before Friday at 11:00am.  They say it is impossible to plan a maintenance day if the jobs haven't been submitted.

Since it can take over an hour to input all of the FACET jobs into the system and there are meetings between 8am and 11am for the FACET User Manager to attend, the FACET User Manager needs to deal with them on Thursday before she goes home. So it is greatly appreciated if she receives them in good time.

A normal week would look like this:
Friday 11:00 Area Managers plan next week's access schedule
Monday 12:00 AOSD Maintenance Office locks the PAMM so no new jobs can be added
Monday 13:00 PAMM planning meeting

Although they don't lock the job list until noon, this doesn't give the Area Managers much time to sort though and approve jobs before the planning meeting. This can be difficult in areas like Linac West where there may be upwards of 50 jobs on a single PAMM day and that is a lot for one person to track.  

 

Question:
​What training do I need to work in the FACET tunnel?

Answer:
See the FAQ on Training.

 

Question:
​What time of day will I do my work?

Answer:
There are usually multiple groups doing multiple jobs in the FACET tunnel. Without staggering jobs, the working area can get congested. This does not lead to efficient or indeed safe work (particularly when there is a pandemic). Therefore, we try to coordinate the jobs to different times of the day. There are also hazards involved with some of the jobs that we don't wish to expose people to unnecessarily so these jobs are scheduled quite strictly and the number of people in the tunnel controlled.

As a result, it is hard to tell until all jobs are submitted the exact schedule for the day. It may not be known until the Monday prior to the access exactly what the script is though every attempt will be made to have a broad plan on the Friday prior to the access.

We may request that people work on an owl shift (midnight - 8am) in some circumstances.

Example PAMM schedule here.

 

Question:
​How do I use the google doc spreadsheet?

Answer:
The google doc spreadsheet can be found here.

https://docs.google.com/spreadsheets/d/1rjzsqHq0zmNjhHckBTkhxspWMRuxko8yIz4nt06-bZ0/edit?pli=1#gid=2146661250

You can see the dates of upcoming planned accesses (subject to change).

If your job is listed, please make sure that all the details are correct. Please feel free to update as long as you don't change any columns with red font. Red font is for use of Christine only! We need coarse estimates for planning our run and accurate inputs by the Thursday before the scheduled PAMM (see other FAQs for more information on why we need this by Thursday).

Any updates after the Thursday before the scheduled PAMM need to be communicated to Christine Clarke by email.

If your job is not listed, please email Christine Clarke to get it on the list with all the details needed to populate the spreadsheet.

Browser/system requirements for the spreadsheet are here:

https://support.google.com/drive/answer/2375082?hl=en

If you can't provide a link to the procedure, please send it by email instead.

 

Question:
​I just want to go down for a look- what do I need to do?​

Answer:
​If you are just looking and not performing any work, you just need training. If you are current in AD103 (FACET Tunnel Hazards/ on-site orientiation) and course 116 (Radiation Worker) then you don't need specific work release. However, it is best to tell Christine of your intent so email/call/add to the google spreadsheet or otherwise communicate this to Christine. Christine will then ensure that you are informed of the work schedule, possible hazards and PPE requirements. This will save time so you don't go to FACET and then realise that the work that is going on precludes your visit.

 

 


TUNNEL ACCESS

Question:
What are the different FACET tunnel access modes?

Answer:
There are two FACET tunnel access modes (Permitted & Guarded. There is no controlled access in FACET):

1). Guarded Access with Radiation Physics Escort - this is designed for short (less than 30 minutes) accesses for one or two experimenters. This access mode minimizes the beam time lost because by guarding the entrance there is no need for a search after the access.

2).  Permitted Access - this is for when longer access is needed or more thanone or two experimenters need to go into the tunnel. This access mobe is used for PAMM. If such an access is required during run time, the experiment may lose as much as a shift of beam so the decision to go in is not done lightly.

More information on the these access modes, how they are done and the impact is found here.

 

Question:
​How do I make an access during a shift?

Answer:
This constitutes a change in program.

If your program changes, you must contact Mark Hogan immediately (and/or Christine Clarke).

Once approved, follow the steps below.

​1. Plan it. What do you want to do? Make sure you know step by step and have all of the parts to hand. Put the plan in the elog.

2. Estimate time. How long do you require?

->  If it is <30 minutes, it can be done under guarded acces. Otherwise, it needs to be done under permitted access.

3. Tell the EOIC that you need an access. Tell them what time you want the access, how long it will last and what type of access you want. The EOIC may advise you that other times will be easier to support and you may enter negotiations!

4. Ask the EOIC to call RPFO so they can survey. (RPFO need at minimum 2 hours warning).

5. Remember to turn the beam off an hour before you need to go in!

6. Go in. Access procedure is here.

7. Whilst in access, it is very important to let MCC know what is going on. Otherwise your guard will leave when time is up and we will go into permitted access and a search will be necessary. This will cause many hours of delay. Best thing to do is to have someone in the trailer to field calls between the tunnel and MCC. No one except this dedicated person should call the tunnel for "updates" because this can delay work in the tunnel.

 

Question:
​What do I do if I enter under guarded access and then discover the scope of the job is larger than I thought?

Answer:
If you are in the tunnel under a guarded access and the job looks like it will take longer than 30 minutes, phone MCC x2151.

Tell them that you wish to go to permitted access and give your new time estimate.

 

Question:
​What do I do if something isn't as it should be in FACET tunnel?

Answer:
​For example, lights are flashing or there is water on the floor or something appears broken.

Call MCC x2151 and report this.

Depending on the issue, you may also want to tell the Area Manager Al Owens, the Experiment Coordinator Christine Clarke or the safety officer Joe Kenny. MCC operators can help you decide whether more people need to be informed and usually make the phone calls.

 

Question:
​How do I access FACET during downtime?

Answer:
When FACET is in downtime it is usually but not necessarily in permitted access. We may lock FACET for reasons such as electrical outages, safety system testing or for certification purposes. In addition, there may be activities going on inside FACET that are incompatible with general access such as electrical work or class 4 laser work.

Please see information below about entering and working in FACET during the downtime assuming FACET is in permitted access.

Do I have to be RWT1 or GERT?
--------------------------------------

FACET is usually a Radiation Area. You need to have RWT1 (up-to-date!) to enter. You need to sign the area RWP.

Remember that anything taken out needs to get surveyed.

It may be de-posted to a RCA in some circumstances. Please pay attention to the signage and remember your GERT and RWT safety training. Even when it is deposted, you need to be RWT trained to do any work. GERTs can enter FACET but they cannot handle any material when FACET is deposted to an RCA.


Is Area Manager release needed or not?
-----------------------------------------------

Green work is released in FACET if you...

– Always follow the AHA (displayed at the entrance),
– Remember your training (you should have course AD103 or equivalent training in FACET hazards or be escorted by someone that has),

– Observe ongoing activities in the area in the event hazards may be present which require your attention.

You do not need further release from the Area Manger if this is followed.

 

Yellow work (non-observational work/work with tools):

Come to Christine Clarke if you have any non-green work to perform. Work that is done in FACET still requires Area Manager approval. Cell phones are listed in the directory if you are on the SLAC network (use VPN if you are offsite or on the visitor net).

Non-routine tasks or tasks with safety risks need a procedure. See separate question on procedure writing.

All "yellow" work in FACET is documented by Christine in the CATER system. It also needs a radiological review so please let me know how many people for how many hours.

Downtime jobs should be submitted as soon as possible. At the very latest, the Thursday before the week in which you want to do the work. Depending on the job, it may or may not be easy to schedule.

There will be a meeting before work gets started for work release and safety briefings.
 

Tours:
Large groups of visitors, tour groups, photo opportunities and equivalent activities require a release from Al Owens.

References:
Work Planning and Control training (course 120):

https://www-internal.slac.stanford.edu/wpc/

Access Requirements (GERT 115 and RWT1 116 training):
http://www-group.slac.stanford.edu/esh/eshmanual/references/radReqWorkEntry.pdf​

 

 


HARDWARE

Question:
What type of motor should I use?​

Answer:
A stepper motor.
 
DC motors always have some sort of encoder or tachometer that feeds back  position information to a controller that commands the DC motors position. Encoders all contain semiconductors of one sort or another and any semiconductors that have been place close to the beam have always failed within  a couple of weeks.  A tachometer that  uses magnetic elements to produce pulses could work- a Hall sensor would be trouble.
 
Stepper motors on the other hand are open loop and rely on precision manufacturing to obtain their accuracy per step and have no feedback for position. Hybrid stepper motors also contain magnets but at this point magnetic failure due to radiation has not been a problem.  In a high radiation environment, if your experiment can work with the stepper motor error specifications it is best to go with the stepper.

 

Question:
What cameras are at FACET​?

Answer:
Currently FACET fully supports:

http://www.alliedvisiontec.com/us/products/cameras/gigabit-ethernet/manta/g-095c.html

 Very likely all Mantas and Prosilicas listed in the website are supported by areaDetector. Besides the Manta 095B we use commonly at FACET, we are using the Manta G146 and the Prosilica GX1050 in LCLS.
 
There are cameras from other vendors also supported by areaDetector. They are listed here: http://cars9.uchicago.edu/software/epics/areaDetector.html
 
More details about areaDetector: http://cars9.uchicago.edu/software/epics/areaDetectorDoc.html