Transcript Identification of Causal Variables for Building Energy

2nd Workshop on Domain Driven Data Mining, Session I： S2208
Dec. 15, 2008
Palazzo dei Congressi, Pisa, Italy
Identification of Causal Variables
for Building Energy Fault Detection
by Semi-supervised LDA
&
Decision Boundary Analysis
Keigo Yoshida, Minoru Inui, Takehisa Yairi, Kazuo Machida
(Dept. of Aeronautics & Astronautics, the Univ. of Tokyo)
Masaki Shioya, and Yoshio Masukawa
(Kajima Corp.)
1
2
Main Point of the Presentation
We propose …
A Supportive Method for Anomaly Cause Identification
by
Combining Traditional Data Analysis
and Domain Knowledge
Applied to Real Building Energy Management System (BEMS)
Root cause of energy wastes was found successfully
3
Outline
 Introduction
 Theories
 Experiments for Real Data
 Conclusions
4
Introduction: What is BEMS ?
 Building Energy Management Systems
 Collect/Monitor Sensor Data in BLDG
(temperature, heat consumption etc…)
 Energy-efficient Control
 Discover Energy Faults (wastes)
I/F
BEMS
5
Introduction: Problem of BEMS
 Hard to identify root causes of Energy Faults (EF)
 Complex Relation between Equipments
 Data Deluge from Numerous Sensors
(approx. 2000 sensors, 20000 points for 20-story)
 Current EF Detection:
Heuristics Based on Expert’s Empirical Knowledge,
usually fuzzy “IF-THEN” rules.
“Heuristic Diagnostics is Incomplete”
 Fuzziness
 Detection-Only
False Negative Error
Cannot Improve Systems
6
Early Fault Diagnosis Methods
Performance
Knowledge-Based
Modeling-Based
• FTA/FMEA
• Bayesian
Filtering
• FDA…
Data-Driven
• Feature Extraction
• Neural Networks…
Expert System
Fuzzy Logic
Supervised Learning
Experts
Easy
Expensive
Poor
Unsupervised
Learning /
Data Mining
Source
Interpretation
Modeling Cost
Versatility
Knowledge Acquisition Bottleneck
Data
Hard
Low
High
Neglecting Useful Knowledge
7
Proposed Method
Performance
Knowledge-Based
Modeling-Based
Data-Driven
Expert System Proposal
Unsupervised
Fuzzy Logic Domain Knowledge
+
Learning /
Supervised Learning
Data Analysis
Data Mining
Experts
Easy
Interpretation:
Expensive
Cost:
Poor
Versatility:
Performance:
- Characteristics -
Data
Source
Hard
Interpretation
exploit domain
knowledge
Low
Modeling Cost
not so high,
empirical
knowledge
only
High
Versatility
easy to apply to various domains & problems
better than heuristics
8
Conceptual Diagram
Learning Boundary
Experts Detection Rule
e.g.
Feedback
Variable
Identification*
* Assumption
Contribution to EF
Incomplete heuristics surely
represent abnormal phenomena
DBA
Data Distribution
Acquire Reliable Labels
with Given Rule
Semi-supervised LDA
Variable #
9
Outline
 Introduction
 Theories
 Semi-Supervised Linear Discriminant Analysis
 Decision Boundary Analysis
 Experiments for Real Data
 Conclusions
10
Semi-supervised LDA
Learning Boundary
Data Distribution
Acquire Reliable Labels
with Given Rule
11
Manifold Regularization [M. Belkin et al. 05]
 Regularized Least Square
Squared loss
for labeled data
Penalty Term
(usually squared
function norm)
Labeled data only
12
Manifold Regularization [M. Belkin et al. 05]
Labeled data only
 Regularized Least Square
Squared loss
for labeled data
 Laplacian RLS:
Squared loss
Penalty Term
(usually squared
function norm)
Penalty Term
Additional term
for intrinsic geometry
Use labeled & unlabeled data
Assumption:
Geometrically close
⇒ similar label
: graph Laplacian
13
Semi-Supervised Linear Discriminant Analysis (SS-LDA)
 LDA seeks projection for small within-cov. & large between-cov.
Between-class
Within-class
 Regularized Discriminant Analysis:
[Friedman 89]
Regularizer
 Semi-Supervised Discriminant Analysis (SS-LDA):
14
Decision Boundary Analysis
Learning Boundary
Data Distribution
Acquire Reliable Labels
with Given Rule
Semi-supervised LDA
Decision Boundary Analysis
 Feature Extraction method proposed by Lee & Landgrabe
C. Lee & D. A. Landgrabe. Feature Extraction Based on Decision Boundary, IEEE
Trans. Pattern Anal. Mach. Intell. 15(4): 388-400, 1993
Class 2 Learned Class 1
Boundary
Top view
Cross-section view
Normal vec.
: disciminantly informative
 Extract informative features from
: discriminantly redundant
normal vectors on the boundary
15
16
Decision Boundary Feature Matrix
Linear:
Nonlinear:
 Define responsibility of each variables for discrimination
17
Outline
 Introduction
 Theories
 Experiments
 Application to Energy Fault Analysis
 Conclusions
18
Energy Fault Diagnosis Problem
EF: Inverter overloaded
Detection Rule
6h M.A. of Inverter output = 100
… but I don’t know the cause
cold
Inverter
hot
coil
Air Handling Unit
humidity
EF
19
Energy Fault Diagnosis Problem
EF: Inverter overloaded
Detection Rule
6h M.A. of Inverter output = 100
EF
… but I don’t know the cause
DATA
cold
&
hot
RULE
Inverter
coil
Air Handling Unit
Find out
root cause of inverter overload
humidity
Energy Fault Diagnosis - Settings
 Air-conditioning time-series sensor data for 1 unit
 instances: 744
 Labeled sample: 10 for each (3% of all)
(based on probability proportional to distance from boundary)
 Hyper-parameters:
NN = 5,
 13 attributes, all continuous
1. Supply Air (SA) Temp.
2. Room Tempe.
8. Humidifier Valve Opening
9. Return Air Temperature
3. Supply Air Temp. Setting
10. Pressure Diff. between In-Outside
4.
5.
6.
7.
11. Moving Ave. of Pressure Difference
12. Outside Air Temperature
13. Outside Humidity
Room Humidity
Inverter Output
Cooing Water Valve Opening
Hot Water Valve Opening
20
Experimental Results
21
22
Results (100 times ave.)
Contribution Score [%]
0
20
SA Temp.
Room Temp.
SA Setting
Room Humidity
Inverter
Inverter
Cooling Water
Hot Water
Humidifier
Return Air Temp.
Pressure Diff.
MA. Pressure
Outside Temp.
Outside Humidity
<LDA>
Inverter (96%)
40
60
80
100
LDA
Trivial
23
Results (100 times ave.)
Contribution Score [%]
0
20
SA Temp.
Room Temp.
SA Setting
Room Humidity
Inverter
CoolingWater
water
Cooling
Hot Water
Humidifier
Return Air Temp.
Pressure Diff.
MA. Pressure
Outside Temp.
Outside Humidity
<LDA>
<SSLDA>
Inverter (96%) Cool water (75%)
SA temp. (12%)
40
60
80
100
LDA
SSLDA
24
Results (100 times ave.)
Contribution Score [%]
0
20
40
60
SA Temp.
Room Temp.
SA Setting
Room Humidity
Inverter
Cooling Water
Hot Water
Humidifier
Return Air Temp.
Pressure Diff.
MA. Pressure
Outside Temp.
Outside Humidity
80
100
Not Distinctive !
LDA
SSLDA
KDA
<LDA>
<SSLDA>
<KDA>
Inverter (96%) Cool water (75%) Cool water (19%)
SA temp. (12%) MA. Pressure (15%)
Inverter
(15%)
…
25
Results (100 times ave.)
Contribution Score [%]
[1]
0
20
40
60
80
100
SA
SA Temp.
Temp.
Room Temp.
[2]
SA Setting
Setting
SA
Room Humidity
Inverter
Inverter
[3] Cooling
Cooling water
Water
Hot Water
Humidifier
Return Air Temp.
Pressure Diff.
MA. Pressure
Outside Temp.
Outside Humidity
LDA
SSLDA
KDA
SSKDA
…
<LDA>
<SSLDA>
<KDA>
<SSKDA>
Inverter (96%) Cool water (75%) Cool water (19%) Inverter
(33%)
SA temp. (12%) MA. Pressure (15%) SA temp (19%)
Inverter
(15%) Cool Water (17%)
SA setting (13%)
26
Energy Fault Diagnosis: Examine Row Data
 Cooling water valve Opening [3]
valve opens completely, but this is result of EF, not cause
27
Energy Fault Diagnosis: Examine Row Data
 Cooling water valve Opening
valve opens completely, but this is result of EF, not cause
 SSLDA/SSKDA
show SA temp. [1] & setting [2] responsible
To reduce this deviation…
• Operate inverter at peak power
• Open cooling water valve
deviation of SA temp.
28
Evaluation
Root Cause
SA Temp.
SA Setting
LDA
SSLDA
KDA
SSKDA
29
Outline
 Introduction
 Theories
 Experiments for Real Data
 Conclusions
30
Conclusions
 Introduce identification method of causal variables
by combining semi-supervised LDA & DBA



Labels are acquired from imperfect domain-specific rule
SS-LDA/SS-KDA: reflect domain knowledge & avoid over-fitting
DBA: extract informative features from normal direction of boundary
 Apply to energy fault cause diagnosis
 Succeeded in extracting some responsible features
beginning with fuzzy heuristics based on domain knowledge
31
Room for improvements
 Consider temporal continuity
 Time-series is not i.i.d.
 Find True Cause from Correlating Variables
32
Thank you for your kind attention
33
Discussions
34
Minor improvements
 Optimize Hyper-parameters
 AIC, BIC, …
 Cross Validation
 Regularization Term
 L1-norm will give sparse solution
 Comparison to other discrimination methods
 SVM
 Laplacian SVM… etc.
35
Extension to Multiple Energy Faults
 In real systems, various faults take place
 Fault cause varies among phenomena
 Need to separate phenomena and diagnose respectively
<Our Approach>
1. Extract points detected by existing heuristics
2. Reduce dimensionality and visualize data in low-dim. space
3. Clustering data and give them labels
4. Identify variables discriminating that cluster from normal data
36
Experimental Condition & Results




Air-conditioning sensor data, 13 attributes, same heuristics
748 instances, operating time only (hourly data for 2 months)
137 points are detected by heuristics
Reduce dimensionality by isomap [J.B. Tenenbaum 00] (kNN = 5)
 Contribution score is given by SS-KDA (kNN = 5,
<2D representation>
2 major cluster,
4 anomalies
)
38
Contribution score for red points
Experimental Condition &Room
Results
air Temp.




superficial
Air-conditioning sensor data, 13 attributes, same heuristics
748 instances, operating time only (hourly data for 2 months)
137 points are detected by heuristics
Reduce dimensionality by isomap [J.B. Tenenbaum 00] (kNN = 5)
 Contribution score is given by SS-KDA (kNN = 5,
<2D representation>
2 major cluster,
4 anomalies
Deviation of Room Air Temp.
around detected points
Detected, this is EF
)
39
Data Distribution
Properly Controlled
System Deviation
40
Cooling Water Valve [%]
Data Distribution
Linearly Separable
for Cooling Water Valve [3]
41
Probabilistic Labeling
Rule
 Points distant from boundary are
reliable as class labels
 Keep robustness against outliers
outlier
Unreliable
Points are stochastically given labels
based on reliability
: Distance from boundary of point
42
Estimate DBFM
 Linear Case:
 Nonlinear Case
Difficult to acquire points on boundary & calculate gradient vector
Disciminant function is linear in feature space
Input space
Kernelized SSLDA
(SS-KDA)
Feature space
43
DBFM for Nonlinear Distribution (1)
1. Generate points on boundary in feature space
Feature space
2. Gradient vector at corresponding point
for Gaussian kernel
Input space
But to find pre-image
is generally difficult…
By kernel trick, pre-image problem is avoidable
44
DBFM for Nonlinear Distribution (2)
Finally we have gradient vectors on boundary for each point
3. Construct estimated DBFM
 Define responsibility of each variables for discrimination
Max. eigenvalue
質問されそうなこと
 リアルタイム性は？
 事後処理を想定
 他の手法と比較したか？なぜLDAか？
 SVMでも適用できるので試したい
 なぜこういう結果になったのか
 原因変数のデータを見ると線形判別は難しい
45