Ben's Comp Newsletter: Issue 029
|
|
Hey,
I want to start this issue with a big congratulations to Dneg and the First Man VFX team for their Oscar win last night! All the nominees this year created some incredible work, and it's great to see this being recognized. Now, onto your regularly scheduled newsletter!
We can agree that Roto is one of the most tedious tasks we need to endure as a Compositor. Ever since I joined the VFX industry in 2011, I've dreamed of having my roto tasks automated. Recent developments have shown this dream might actually become a reality in the near future!
I've briefly touched on advancements in this area in previous issues of Ben's Comp Newsletter, although I felt it was worth diving deeper, to explore the idea and progress of "Automating Roto", and get a sense of when it could be at a production-ready quality...
|
|
Rotobot
Sam Hodge, the mind behind Kognat, has been developing "Rotobot" for Nuke. It works via implementing a deep learning algorithm called "semantic segmentation", which assigns pixels to a certain "class", (e.g. "person", "car", etc.). Rotobot was recently featured in an in-depth article over on fxguide, which is an insightful read!
|
|
Mask R-CNN
Similar to Rotobot, Mask R-CNN "automatically segments and constructs pixel-wise masks for every object in an image". This article breaks down the difference between Image Classification, Object Detection, Instance Segmentation and Semantic Segmentation, and how it applies to Mask R-CNN.
The algorithm appears to be broken down into the following steps:
- Analyse the image, and draw bounding boxes around individual objects
- Refine bounding boxes
- Generate masks for objects inside bounding boxes
- Debug, via "activating" layers, inspecting "weight histograms" & logging to "Tensorboard"
- Compose different pieces into a final result
This article gives a different perspective on the same algorithm, and offers some more insight into what's going on under the hood.
If you're interested in giving this a go yourself, click the button below!
Thanks to Miles Lauridsen for the tip -- he conveniently compiled OpenCV in a Docker Ubuntu image, which may help you get started faster!
|
|
Adobe's #ProjectFastmask
Shared in Ben's Comp Newsletter: Issue 024 was #ProjectFastmask. It's worth revisiting, as Adobe likely has the most resources available to tackle solving the "Automated Roto" pipe dream, and therefore might have the best opportunity to crack the case!
As shown in their demo video, Adobe's results aren't quite production-ready at this point, but their results are amongst the most promising!
|
|
Deep Learning: A Crash Course
If you're intrigued and want to learn more about Deep Learning, Andrew Glassner gave a talk last August on the topic. Andrew offers insights on how the technology works, and the implications it has on the computer graphics industry.
Thanks again to Sam Hodge for the tip!
|
|
Did you find this newsletter informative?
Have you created, or do you know of any outstanding Gizmos, Python Scripts or Tutorials that you would like to share with the global Compositing community? Please send me an email, and I will do my best to include it in a future issue of this newsletter.
If you find value in Ben's Comp Newsletter, please consider pledging a small amount on Patreon to help keep this project running! Contributions are always appreciated but not expected. |
|
|
|
|
|
