Transcript On the performance of three deep-diving underwater gliders

Risk and Reliability …
Towards Targeted Reliability
Oceans 2025 Theme 8 WP8.4
Gwyn Griffiths and Mario Brito
National Oceanography Centre, Southampton
2008 Moorings Workshop
Is there common ground between people
working on moorings and those working on
improving the reliability of Autonomous
Underwater Vehicles?
Risk and Reliability …
Nick Millard
Fimbul Ice Sheet, Antarctica
Risk and Reliability …
Inquiry following loss of Autosub2
“The Board has recognised the competence and
commitment of the NOC AUV team; they have a high level
of understanding of the importance of reliability and have
employed sound reliability principles to influence their
design decisions. However, they have not employed any
formal systems reliability analysis methods. The Board
believe this to be a major shortfall.”
Risk and Reliability …
Recommendations following loss of Autosub2
• NERC (or representatives) should define risk
acceptance criteria
• AUV development team should implement formal
risk and reliability management systems
• AUV development team should provide evidence
of reliability achievement
Full report available:
Strut, J. (editor), 2006. Report of the inquiry into the loss of Autosub2 under the
Fimbulisen. NOCS Research and Consultancy Report: pdf at
http://eprints.soton.ac.uk/41098/
Risk and Reliability …
Synopsis
 Introduction to Targeted Reliability in a marine science
context
 Risk Management Process-AUV being used with Autosub
 Informing the process
 Detailed engineering and operational fault logs
 Engineering and statistical follow-up
 The use of Expert Judgement
 Moorings
 What might a Risk Management Process look like?
 Example issues
Risk and Reliability …
Risk Management Process-AUV
Start
e.g.
P(loss)<20%
Risk and Reliability …
Campaign requirements, Dr Jenkins (BAS) for
Pine Island Glacier, Antarctica Feb. 2009
60 km open water missions to 200, 600, 1000 m
depths, close to ice front.
Minimum
3 x 60 km sub-ice-shelf missions to 600 m depth, in
outer half of cavity.
3 x 120 km sub-ice-shelf missions to 1000 m depth, Desired
to the "minimum headroom" limit of the cavity
Sea ice may well be present in the area, beneath which
Autosub3 would need to travel to reach the ice front.
Risk and Reliability …
How might we predict probability of loss?
*
Part A - Gather fault history, document human error and all
incidents with the AUV
Part B - Set out the key features and risks of the operating
environment
We postulate that combining Parts A and B cannot be done through
scientific methods. Addressing Part A alone has been controversial.
* Estimated as cause of ~60% of US military UAV faults/incidents by Tvaryanas et al. (2005)
Risk and Reliability …
Part A: Example Autosub3 fault log entries
Mission
384
Distance (km)
1.5
No. Fa ults
2
Fault HIU?
Y(1)
385
15.2
1
N
386
26
1
N
Comment
1. Mission aborted due to network failure. (Much)
later tests showed general problem with the
harnesses (bad crimp join ts). Harnesses repaired by
manufacturer for next cruise.
2. Loop o f recovery line came out from storage slot,
New storage arrangement needed.
GG not e: the Terschelling 2006 trials showed no
explicit evidence of harness-connector problems.
1. Autosub headed off in an uncontrolled way. This
was due to a side effect of the remo val of the
upwards-looking ADCP. The SW vulnerability that
caused this was corrected for later test cruise. This
problem wo uld be caught immediately after launch
and before the vehicle would be committed to its
mis sion.
1. GP S antenna failed. New design for future
cruises.
GG not e: new GPS antenna design used, but
differen t set of problems encountered on
Terschelling 2006.
Risk and Reliability …
Autosub Engineering mitigation examples
 Replace wet-mateable connectors with
penetrators - prior experience of intermittent
connection under pressure.
Autosub2 mission 313 Amundsen Sea
Risk and Reliability …
Part B. Eliciting Expert Judgement
1.
Set out the Issues
2.
Select the Experts
3.
Clearly Define the Issues
4.
Train the Experts
5.
Elicit the Judgements
6.
Analyze and Aggregate the Results
7.
Complete Analysis and Write-up
Otway, H. and von Winterfeldt, D., 1992. Expert judgement in risk analysis and
management: Process, context, and pitfalls. Risk Analysis, 12(1): 83–93.
Risk and Reliability …
EEJ example for the fault history of Autosub3
Ten AUV practitioners from Australia, Canada, USA from academia,
research, commercial, military backgrounds.
q
Given the set of facts on all faults and incidents with Autosub3
throughout its life to date we seek to predict the probability of loss of
the vehicle in four operating environments:
> Open water
> Coastal
> Sea ice present
> Under an ice sheet
1. In the course of evaluating each fault log entry, the expert is asked
to assess the following question:
“What is the probability of loss of the vehicle in the given
environment X given fault/incident Y?”
Risk and Reliability …
Your estimates for Autosub3 Mission 384
Fault/incident description
Open
Mission aborted (to surface) 0.001 0.01
due to network failure. 0.001 0.01
(Much) later tests showed 0.01 0.001
general problem with the 0.01
harnesses (bad crimp joints).
Weights 1-5
Range
Coast
0.001 0.015
0.001 0.005
0.01 0.004
0.03
Sea Ice
0.01
0.01
0.1
0.1
0.5
0.1
0.05
Shelf Ice
0.7
0.1
0.7
1.0
0.8
0.5
0.8
4, 2, 5, 4, 5, 3, 4 4, 2, 5, 4, 5, 4, 3 4, 3, 5, 3, 3, 2, 3 4, 3, 4, 3, 2, 1, 4
0.001 - 0.01
0.001 - 0.03
0.01 - 0.5
0.1 - 1
Log Opinion Pool
0.0037
0.0051
0.0590
0.5523
Weighted Log Opinion Pool
0.0036
0.0043
0.0472
0.5056
Risk and Reliability …
Experts’ estimates for Autosub3 Mission 384
Fault/incident description
Open
Mission aborted (to surface) 0.001 0.01
due to network failure. 0.001 0.01
(Much) later tests showed 0.01 0.001
general problem with the 0.01
harnesses (bad crimp joints).
Weights 1-5
Range
Coast
0.001 0.015
0.001 0.005
0.01 0.004
0.03
Sea Ice
0.01
0.01
0.1
0.1
0.5
0.1
0.05
Shelf Ice
0.7
0.1
0.7
1.0
0.8
0.5
0.8
4, 2, 5, 4, 5, 3, 4 4, 2, 5, 4, 5, 4, 3 4, 3, 5, 3, 3, 2, 3 4, 3, 4, 3, 2, 1, 4
0.001 - 0.01
0.001 - 0.03
0.01 - 0.5
0.1 - 1
Log Opinion Pool
0.0037
0.0051
0.0590
0.5523
Weighted Log Opinion Pool
0.0036
0.0043
0.0472
0.5056
Risk and Reliability …
Cumulative frequency statistics
Open Water
Coastal
Sea Ice
Shelf Ice
Upper Q.
.026
.037
.17
.40
Median
.018
.020
.088
.17
Lower Q.
.0085
.0083
.045
.072
Risk and Reliability …
Autosub Statistical (procedural) example
1.0
0.9
0.8
Surviving
0.7
0.6
0.5
0.4
0.3
Operating procedure for under ice:
Each mission has ‘open water’ 25km
segment before committing under ice.
0.2
0.1
Only two
missions
beyond
250km, one
of which
failed,
hence large
step.
0.0
0
50
100
150
200
Distance(km)
250
300
Kaplan-Meier method for estimating probability of survival with distance for all Autosub3
missions to date. Prevalence of faults leading to ‘infant mortality’ using GG’s judgement.
Risk and Reliability …
Estimated probability of loss
Minimum mission set, no sea ice in front of glacier P(loss) = 9%
Minimum mission set + 30km of sea ice in front of glacier
P(loss) = 16%
Desired mission set with no sea ice in front of glacier
P(loss) = 24%
Desired mission set + 30km of sea ice in front of glacier
P(loss) = 30%
Based on Autosub3 history to end of March 2007. Will be
updated after Terschelling June 2008 proving trials.
Risk and Reliability …
Risk and Reliability: A new service to the
NERC marine science community
 Part of Oceans2025 Technology Work Package on
Risk and Reliability
 Mario Brito - (awaiting) PhD in software reliability
 Access to engineering specialists in the Underwater
Systems Lab.
 Early discussions on PAP mooring-related problems have
already taken place.
For no-fee consultations on risk and reliability issues for NERC marine science,
contact either:
Mario Brito ([email protected]) or Gwyn Griffiths ([email protected]).
http://www.noc.soton.ac.uk/OED/gxg/risk.html
Risk and Reliability …
Example - Flooded Glider, April 2008
Establish root cause of glider partially flooding
with ~4 litres of water while on a tethered
dockside post incident-free 3-month
deployment.
Depth 0 to 7 m
1 min
Pressure reduction
9.5 to 6.0 in Hg
Risk and Reliability …
Systematic Fault Tree Analysis
CTD
Stern tube
Leak
Ammonite
flooded
‘O’ rings
Chance
events
Failure
routes
Establish actual
cause or assign
probabilities. Do
NOT jump to
conclusions!
Pressure port
Vacuum plug
Pressure tubes
separated
Li battery
vented
Tie-rod cross
threaded
Pressure tubes
not fully butted
Tapered ring
in wrong order
Risk and Reliability …
The root cause: Assembly error
Courtesy Peter Stevenson
For no-fee consultations on risk and reliability issues, contact either:
Mario Brito (mpb2o07 @noc.soton.ac.uk) or Gwyn Griffiths ([email protected]).
Risk and Reliability …
Glass buoyancy failure: WHOI
VEX Mooring Array in ca. 5000m
Western Atlantic.
Where can I find quantitative:
• Failure rates?
• Any difference in failure rates
between brands e.g. Benthos,
Vitrovex?
• The major causes of failure?
• What’s worse and why, time at
depth or depth cycling?
Risk and Reliability …
Acoustic release failure: WHOI VEX
Mooring Array in ca. 5000m Western
Atlantic.
Batch of EG&G releases had improperly
machined release mechanisms. Below
2000m, compression was such that the
mechanism would never release.
Example of a common mode, human error.
Recovery using Isis ROV, April 2003.
Risk and Reliability …
Work programme for 2008
 Analyze and write up Autosub3 Expert Judgement
 Work with Autosub3 team on 2008 trials and risk management
for 2009 Antarctic cruise
 How do we incorporate quantitatively sea ice and vessel
characteristics?
 Paper for IPY Conference, St. Petersburg July 2008.
 Work with Autosub6000 team on Markov chain approach to stages
of reliability and risk.
 The reliability of deep ocean glass spheres.
 Factors affecting the reliability of the PAP moorings
 Related to EuroSITES project and Oceans 2025 - propose to
instrument a test mooring to establish in situ performance.
 Discuss way forward for interaction with Rapid-Watch with
NERC/Coordinator/Scientists
Risk and Reliability …
Conclusions
 From open literature searches, AUV community lags the
UAV community in analysis of incident and fault data. We
should be more proactive, e.g. use of wikis, blogs, list
servers …
 Recording fault and incident data, and sharing outcomes
is important for the community as a whole
 Controversy still surrounds attempts to model statistically
AUV faults, and more so, and the use of expert judgement
to estimate probability of loss from fault history.
 We need to do more to engage with ocean engineers
working on moorings, landers etc.