2021-04-06 2021-04-06

The new environment for Picam360 includes higher performance and higher quality video conversion processing, server-side software and viewer applications that are easier to install, and a video delivery web application.

Hello, my name is GAKU from the Picam360 development community.

We haven’t updated articles at all lately, as we’ve been focusing on developing new softwares. I apologize to those who have been reading our blog.
What we’ve been doing while we haven’t been updating is that we’ve spent about two years improving Picam360’s video conversion process to produce higher quality output videos. In addition, the server-side software and viewer application had been completely redesigned, and a new video distribution web application had been added, as well as a new Picam360 and its surrounding environment. The new softwares have been upgraded to a completely new versions that utilize the knowledge gained from the existing softwares. We don’t have all of the software we plan to use yet, but we’re close. The ones that are not ready yet are on track to be released in the next month or two. I’m excited to share them with you.
Well, we have been updating this blog irregularly, but from now on, we will deliver it every week. The contents will focus on news about Picam360 and technical stuff. I hope you will enjoy the articles, which we will create based on the criteria that the content is technically interesting and useful. The blog is starting anew, and there are some articles that have been posted but were suspended (Jetbot, etc.). We are sorry to say that we will no longer be updating them (we may rewrite them at some point), as they use old softwares and the release of the new softwares makes it impossible to continue the content. Thank you for your understanding.

In this first installment of our new blog, we’ll start with an introduction to what Picam360 is all about, including what’s changed with the new softwares.

What is Picam360?

First, let me explain about Picam360. We have not been able to provide enough information about Picam360 until now, so this explanation may help those who are new to Picam360 and also those who already know about it to discover something new.

USB connection type panoramic camera

Picam360 is a camera with a wide-angle lens that captures 360 degrees (hemisphere) in its angle of view. It is connected to a computer via USB. The UVC (USB Video Class) communication standard is a general-purpose standard that does not require any special software, as the UVC camera uses the standard OS drivers.

Here is an example of simply taking a picture with a USB connection.
In this example, a jetson nano is used as the computer, and the procedure is the same for a raspberry pi.
Execute the following commands in the console:

// Install fswebcam to take pictures.
// If you have already installed it, it is not necessary.
$sudo apt-get update
$sudo apt-get install fswebcam

// Make sure the camera is recognized.
// If no other cameras are connected, /dev/video0 will be present
// If another camera is connected, the number at the end will change.
$ fswebcam --list-inputs

// Take a picture.
// In the following example, the image will be saved as test.jpg with size of 640x480 pixels.
$ fswebcam -i 0 -d v4l2:/dev/video0 -r 640x480 --jpeg 85 test.jpg

// Install the software fbi to display the image from the console.
// If you have already installed it, it is not necessary.
$ sudo apt-get update
$ sudo apt-get -y install fbi

// Displays the image.
$ fbi -a test.jpg

In this way, any software that supports webcams can easily acquire images.

However, as you can see, the image taken is a fisheye image itself. There are few situations where a fisheye image is used as is, so when it is actually used, it must be processed in some way to make it look natural to humans. If you want to build your own conversion program and use it freely, that’s fine. On the other hand, there are those who want the conversion process to be provided as a function.

That’s why we offer the Picam360 software suite, which not only converts fisheye images to human-viewable natural images, but also provides a set of features necessary for viewing. Moreover, it has some great features that are not available anywhere else.

Open source software suite

As mentioned in the previous section, Picam360 does not require any special software, but provides a software suite of useful features that can be easily used. A suite is a collection of functionally related software, such as Microsoft’s Office. The Picam360 software suite is not actually one specific package, but rather a collection of related software, which we use the term suite to refer to as a single name.

The Picam360 software suite includes:

  • Video Conversion Software
  • Video Streaming Web Server Software
  • Video Viewer application

In addition, there is another software as a peripheral function. Wouldn’t it be interesting to publish Picam360-equipped devices (e.g., radio-controlled devices) on the web for people all over the world to access and play with, or to be able to play with devices created by others remotely? We have created a place on the web where everyone can share and play together. This will be the subject of another article.

We want to make Picam360 available to as many people as possible, so we are basically offering it as open source and free.

Open Source:

  • Video Streaming Web Server Software
  • Video Viewer application

Proprietary:

  • Video Conversion Software

The not-so-open-source range is video conversion software that compresses video from camera fisheye images. For the video compression technology, we have developed and incorporated our own technology optimized for 360 degree videos. This part of the software is not disclosed to the public, but the rest of the video streaming web server software and the viewer application for viewing the video are open source. Feel free to modify it and incorporate it into your own products. We have already released the source code, but we are not ready to provide explanations and other information, so we would like to introduce it in a future blog post.

Software Features

Ultra-light streaming

In order to display 360 degrees in high quality, it is necessary to shoot a large area in high resolution, which requires a much larger amount of data compared to normal video. The horizontal angle of view for a typical smartphone video is about 60°. If you use that camera to cover 360°, 360° divided by 60° equals 6 cameras!
Picam360 software has been developed to reduce the amount of data while maintaining the high quality of 360° video. Typically, 360 videos are stored in a format called the equirectangular projection. Compared to that existing method, Picam360 software compresses the data size by a factor of 10. This is a bit technical, but as an example, a 360° video with 8K resolution delivered using existing methods has a bit rate of about 20Mbps, but Picam360 software compresses it down to 2Mbps.

data size comparison

Supports ultra-high resolution up to 16K

It supports ultra-high resolution of up to 16K all around. With ultra-high resolution video, you can zoom in to see the details of even distant subjects.

3D 360° video with two cameras (to be released soon)

Live streaming of 3D stereoscopic 360-degree video is possible by connecting two panoramic cameras.

Cross-Platform

Both server devices and viewer applications can be used in a wide range of environments.

Server Device OS
Windows, Mac, Linux, Raspberry Pi

Viewer App

  • Desktop apps: Windows, Mac, Linux,
  • Mobile apps：iOS app, Android app
  • VR apps：Oculus app (In preparation for the public)
  • and Browser app

picam360 apps

Specifications such as maximum resolution, FPS, and the number of cameras that can be connected will vary depending on the hardware. We’ll be posting more information about the specs and how to set them up one by one in future blog posts. In the next post, we will show you how to set up a Jetson Nano as a server device for real-time streaming.

Open Source

You are free to modify the software and incorporate it into your products. The license type is GPL. It is available for commercial use, and does not include strict restrictions such as requiring source code disclosure for redistribution. There are some restrictions, mainly regarding the retention and clarification of copyrights. For more details, please see here.

The software isn’t just for Picam360 cameras

You may be thinking, “Could it be?”

The camera is UVC and the server device is multi-platform, which means that the Picam360 software can be used with more than just the Picam360 camera?

Yes, it is possible. We expect that there will be requests to use ultra-high resolution cameras and faster computers for industrial applications. You can integrate it freely, but the free version is limited to a maximum of 15 FPS and the number of connected cameras is limited to one when using non-Picam360 brand cameras. We sell paid licenses that remove these restrictions. Please contact us for more information.

What has changed with the new software?

So far, I have introduced Picam360 on new softwares basis. The changes compared to the old software are as follows:

  • Improved image quality and frame rate (FPS) by optimizing the video conversion process
  • Windows, Mac, and Linux are added to the supported OS for server-side devices
  • The previous viewer app was only available in a browser version, but we have added mobile and desktop apps for Windows, Mac, Linux, iOS app, Android app, and Oculus app (in preparation for release). By making it a dedicated application, it is possible to get more out of the viewer device and improve the frame rate of the viewer.
  • The Windows, Mac, and Linux versions of the server-side software and viewer apps have been created as Node.js apps, which can be installed using npm, making them easier to install than before.

The above is a summary of the contents of this update. There are some other minor improvements, but it’s a long story, so I’ll skip it.

Well, that concludes the first installment of the New Start Blog. In the next article, I will show you the steps to set up the Jetson Nano and watch videos in the viewer. We hope that you will be able to try it out.

Picam360 Links：