Editor’s Choice: Feb 2022

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

Images, such as illustrations, paintings, and photographs, can often be easily described using text but can require specialized skills and hours of labor to create. Therefore,

a tool capable of generating realistic images from natural language can empower humans to create rich and diverse visual content with unprecedented ease. The ability to edit images using natural language further allows for iterative refinement and fine-grained control, both of which are critical for real-world applications.

Links: https://arxiv.org/pdf/2112.10741v1.pdf

Habitat 2.0

Habitat-Sim is typically used with Habitat-Lab, a modular high-level library for end-to-end experiments in embodied AI — defining embodied AI tasks (e.g. navigation, instruction following, question answering), training agents (via imitation or reinforcement learning, or no learning at all as in classical SensePlanAct pipelines), and benchmarking their performance on the defined tasks using standard metrics.

A high-performance physics-enabled 3D simulator with support for:

· 3D scans of indoor/outdoor spaces (with built-in support for HM3D, MatterPort3D, Gibson, Replica, and other datasets)
· CAD models of spaces and piecewise-rigid objects (e.g. ReplicaCAD, YCB, Google Scanned Objects),
· Configurable sensors (RGB-D cameras, egomotion sensing)
· Robots described via URDF (mobile manipulators like Fetch, fixed-base arms like Franka, quadrupeds like AlienGo),
· Rigid-body mechanics (via Bullet).

Link: https://github.com/facebookresearch/habitat-sim

Hierarchical Neural Story Generation

Abstract: creative systems that can build coherent and fluent passages of text about a topic through hierarchical story generation, where the model first generates a premise and then transforms it into a passage of text.

Link: https://arxiv.org/abs/1805.04833

Faster and Better NLP using PyText from Meta

Abstract: PyText is a library built on PyTorch, an open-source deep learning framework. It offers several benefits for NLP development:

· A simplified workflow for faster experimentation.
· Access to a rich set of prebuilt model architectures and utilities for text processing and vocabulary management to facilitate large-scale deployment.
· The ability to harness the PyTorch ecosystem, including pre-built models and tools created by researchers and engineers in the NLP community.

Link: https://github.com/facebookresearch/pytext?fbclid=IwAR0b03M7Q-dfQNqoWDEsvBh0gedVNCeun9Nvt4hdGmn1dVy4_7Mgf6N2lFM