인간이 프로그래밍을 통해 기계에 모든 행동을 입력하지 않고도 인공지능 스스로 인간이 수행하는 복잡한 일을 수행할 수 있도록 하는 것이 목표라고 할때, 그 목표를 위한 첫걸음은 test bed를 충분히 마련하는 것일 것이다. 이런 공감대에 호응하여, 인공지능 성능을 테스트 할 수 있는 실험실이 마련되어 있고 그것을 일반인에게 공개하고 있다. 그 중 유명한 곳을 아래 소개하였다.
Universe by OpenAI
Universe is a software platform for measuring and training an AI’s general intelligence across the world’s supply of games, websites and other applications. Universe makes it possible for any existing program to become an OpenAI Gym environment, without needing special access to the program’s internals, source code, or APIs. It does this by packaging the program into a Docker container, and presenting the AI with the same interface a human uses: sending keyboard and mouse events, and receiving screen pixels. Our initial release contains over 1,000 environments in which an AI agent can take actions and gather observations.
OpenAI is a non-profit organization that aims to collaborate with the research and industry community, and releasing the results to public for free. It was created in late 2015, and started delivering the first results (publications like InfoGAN, platforms like Universe and (un)conferences like this one) in 2016. The motivation behind it is to make sure that AI technology is reachable for as many people as possible, and by doing so, avoiding the creation of AI superpowers.
Additionally, some environments include a reward signal sent to the agent, to guide reinforcement learning. We’ve included a few hundred environments with reward signals. These environments also include automated start menu clickthroughs, allowing your agent to skip to the interesting part of the environment.
DeepMind Lab by Google
DeepMind Lab provides a suite of challenging 3D navigation and puzzle-solving tasks for learning agents. Its primary purpose is to act as a testbed for research in artificial intelligence, especially deep reinforcement learning.
CommAI-Env by Facebook
CommAI-env is a platform for training and testing an AI system, the Learner (coded in an arbitrary language of the system developer’s choice), in a communication-based setup where it interacts via a bit-level interface with an Environment. The Environment asks the Learner to solve a number of communication-based Tasks, and assigns it a Reward for each task instance it successfully completes.
The Learner is presented, in random order, with multiple instances of all tasks, and it has to solve as many of them as possible in order to maximize reward. Examples of tasks currently implemented include counting problems, tasks where the Learners must memorize lists of items and answer questions about them, or follow navigation instructions through a text-based navigation scheme (see this document for detailed descriptions of the tasks). The set of tasks is open: we are constantly extending it, and we invite others to contribute.
The ultimate goal of CommAI-env is to provide an environment in which Learners can be trained, from ground up, to be able to genuinely interact with humans through language. While the tasks might appear almost trivial (but try solving them in the scrambled mode we support, where your knowledge of English won’t be of help!), we believe that most of them are beyond the grasp of current learning-based algorithms, and that a Learner able to solve them all would have already made great strides towards the level of communicative intelligence required to interact with, and learn further from human teachers.