Documentation for auto_LiRPA¶
Introduction¶
auto_LiRPA is a library for automatically deriving and computing bounds with
linear relaxation based perturbation analysis (LiRPA) (e.g.
CROWN and
DeepPoly) for
neural networks, which is an useful tool for formal robustness verification. We
generalize existing LiRPA algorithms for feed-forward neural networks to a
graph algorithm on general computational graphs, defined by PyTorch.
Additionally, our implementation is also automatically differentiable,
allowing optimizing network parameters to shape the bounds into certain
specifications (e.g., certified defense). You can find a video ▶️ introduction
here.
Our library supports the following algorithms:
Backward mode LiRPA bound propagation with optimized bounds (α-CROWN)
Backward mode LiRPA bound propagation with split constraints (β-CROWN)
Generalized backward mode LiRPA bound propagation with general cutting plane constraints (GCP-CROWN)
Forward mode LiRPA bound propagation (Xu et al., 2020)
Forward mode LiRPA bound propagation with optimized bounds (similar to α-CROWN)
Interval bound propagation (IBP)
Hybrid approaches, e.g., Forward+Backward, IBP+Backward (CROWN-IBP), α,β-CROWN (alpha-beta-CROWN)
Our library allows automatic bound derivation and computation for general
computational graphs, in a similar manner that gradients are obtained in modern
deep learning frameworks – users only define the computation in a forward
pass, and auto_LiRPA traverses through the computational graph and derives
bounds for any nodes on the graph. With auto_LiRPA we free users from
deriving and implementing LiPRA for most common tasks, and they can simply
apply LiPRA as a tool for their own applications. This is especially useful
for users who are not experts of LiRPA and cannot derive these bounds manually
(LiRPA is significantly more complicated than backpropagation).