Zuchen Li

All projects

arXiv preprint coming soon Human–AI interaction

VeilLens

Explainable and Granular Analysis and Protection Against Visual Privacy Risks Beyond Visible Appearance

Co-first author, Human Centered Computing Lab, University of Michigan, with Prof. Ke Sun

Ask a modern vision-language model where an ordinary street photo was taken, and it may answer correctly to within a few hundred meters. Faces and license plates are only the obvious part: today’s models combine weak, everyday clues — a work badge, a logo on a bag, a shop sign, a street bollard, the people you are with — and together those clues reveal who you are, where you are, and who you are with. Most existing tools blur fixed categories or perturb the whole image; they rarely show what leaks, let you decide what to change, or check what still gets through.

VeilLens turns these hidden inferences into evidence you can inspect and edit.

How it works

  • Reason — a trusted vision-language model explains what an attacker could infer, and why.
  • Ground — each clue is localized to an exact image region, with a privacy level and a reason.
  • Edit — the clues the user selects are removed with generative inpainting that keeps the rest of the scene.
  • Verify — a verifier re-attacks the result, and the loop repeats until the inference is suppressed.

Protection is granular: in the demo, geolocation is protected at the street and city level while country-level clues are deliberately kept. VeilLens advises, but the user decides.

Results

  • Evaluated on 1,454 images with more than 20,000 model queries.
  • Compositional privacy risk reduced by 48.7%.
  • Against the strongest attacker tested, street-block geolocation success fell from 40.2% to 14.6%, well below the strongest prior defense.
  • Relationship inference dropped by more than 92%.
  • In a study with 25 participants, 70.7% of choices preferred the protected image, with high ratings for understanding the risk and trusting the protection.

Videos

  1. 30-second teaser 0:30
  2. Narrated demo 3:06

Figures

A vision-language model geolocates an ordinary street photo to Aachen, 0.4 km from the true spot.
Figure 1. A vision-language model geolocates an ordinary street photo to Aachen, 0.4 km from the true spot.
Weak, everyday clues — badges, logos, signs, bollards — combine into identity, location, and relationship inferences.
Figure 2. Weak, everyday clues — badges, logos, signs, bollards — combine into identity, location, and relationship inferences.
The Reason → Ground → Edit → Verify loop.
Figure 3. The Reason → Ground → Edit → Verify loop.
Detected clues, each with a privacy level and a reason, grounded to exact regions.
Figure 4. Detected clues, each with a privacy level and a reason, grounded to exact regions.
A second round finds residual evidence; the user decides what to keep.
Figure 5. A second round finds residual evidence; the user decides what to keep.
After editing, the best guess jumps ~70 km away with low confidence.
Figure 6. After editing, the best guess jumps ~70 km away with low confidence.
Protection summary — only 2% of pixels changed.
Figure 7. Protection summary — only 2% of pixels changed.
Evaluation results.
Figure 8. Evaluation results.