Ben's Comp Newsletter: Issue 029
|
|
Hey,
I want to start this issue with a big
congratulations to Dneg and the First Man
VFX team for their Oscar win last night! All
the nominees this year created some
incredible work, and it's great to see this
being recognized. Now, onto your regularly
scheduled newsletter!
We can agree that Roto is one of the most
tedious tasks we need to endure as a
Compositor. Ever since I joined the VFX
industry in 2011, I've dreamed of having my
roto tasks automated. Recent developments
have shown this dream might actually become
a reality in the near future!
I've briefly touched on advancements in this
area in previous issues of Ben's Comp
Newsletter, although I felt it was
worth diving deeper, to explore the idea and
progress of "Automating Roto", and get a
sense of when it could be at a
production-ready quality...
|
|
Rotobot
Sam Hodge, the mind
behind Kognat,
has been developing "Rotobot" for Nuke.
It works via implementing a deep
learning algorithm called "semantic
segmentation", which assigns pixels to a
certain "class", (e.g. "person",
"car", etc.). Rotobot was recently
featured in an in-depth
article over on fxguide,
which is an insightful read!
|
|
Mask R-CNN
Similar to Rotobot, Mask R-CNN
"automatically segments and
constructs pixel-wise masks for every
object in an image". This
article breaks down the difference
between Image Classification, Object
Detection, Instance
Segmentation and Semantic
Segmentation, and how it applies to
Mask R-CNN.
The algorithm appears to be broken down into
the following steps:
-
Analyse the image, and draw bounding
boxes around individual objects
-
Refine bounding boxes
-
Generate masks for objects inside
bounding boxes
-
Debug, via "activating" layers,
inspecting "weight histograms" &
logging to "Tensorboard"
-
Compose different pieces into a
final result
This
article gives a different
perspective on the same algorithm, and
offers some more insight into what's going
on under the hood.
If you're interested in giving this a go
yourself, click the button below!
Thanks to Miles Lauridsen
for the tip -- he conveniently compiled OpenCV
in a Docker Ubuntu image, which may
help you get started faster!
|
|
Adobe's #ProjectFastmask
Shared in Ben's Comp Newsletter: Issue
024 was #ProjectFastmask. It's
worth revisiting, as Adobe likely has the
most resources available to tackle solving
the "Automated Roto" pipe dream, and
therefore might have the best opportunity to
crack the case!
As shown in their demo video, Adobe's
results aren't quite production-ready at
this point, but their results are amongst
the most promising!
|
|
Deep Learning: A Crash Course
If you're intrigued and want to learn more
about Deep Learning, Andrew
Glassner gave a talk last
August on the topic. Andrew
offers insights on how the technology works,
and the implications it has on the
computer graphics industry.
Thanks again to Sam
Hodge for the tip!
|
|
Did you find this
newsletter informative?
Have you created, or do
you know of any
outstanding
Gizmos, Python
Scripts or Tutorials
that you would like to
share with the global
Compositing community?
Please send
me an
email, and I
will do my best to
include it in a future
issue of this
newsletter.
If you find value in
Ben's
Comp Newsletter,
please consider
pledging
a small amount
on
Patreon to help
keep this project
running! Contributions
are always appreciated
but not expected.
|
|
|
|
|
|
