A Simple Solution for iOS Video Stretching Correction

Introduction

When using Temasys Platform’s (a.k.a. Skylink) iOS SDK for the first time, it’s pretty exciting to see how quickly one can set up and make a video call. However, iOS video stretching is a common problem for video calling apps on iOS. In this article I provide a walkthrough of a simple solution that will help you set up Temasys on iOS, and prevent the user’s video from being stretched.


Setup

Let’s start with a very basic view controller, where we configure  and create  the ‘SKYLINKConnection’ and then connect to a room in ‘viewDidLoad’ :

#import <UIKit/UIKit.h>;
#import <SKYLINK/SKYLINK.h>;
@interface ViewController : UIViewController
@end

#import "ViewController.h"

@interface ViewController ()
@property (strong, nonatomic) SKYLINKConnection *skylinkConnection;
@end

#import "ViewController.h"
@implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];

// Creating configuration
SKYLINKConnectionConfig *config = [SKYLINKConnectionConfig new];
config.video = YES;
config.audio = YES;
config.timeout = 30;

// Creating SKYLINKConnection
self.skylinkConnection = [[SKYLINKConnection alloc] initWithConfig:config appKey:@"YOUR-API-KEY"];

// Connecting to a room
[self.skylinkConnection connectToRoomWithSecret:@"YOUR-SECRET" roomName:@"ROOM-NAME" userInfo:nil];
}

@end

This will connect the user to a room. The front camera is already streaming the user’s video in this room. However, the UI does not reflect this on screen. How boring is that?!


Show me my face!

We now want to show the user his own face on the screen. To do so, we need to declare the ‘viewController’ as a ‘SKYLINKConnectionLifeCycleDelegate’ and set it as the SKYLINKConnection’s lifeCycleDelegate in order to implement the following delegate method : – (void)connection:(SKYLINKConnection*)connection didRenderUserVideo:(UIView*)userVideoView;

</kbd>
#import <UIKit/UIKit.h>;
#import <KYLINK/SKYLINK.h>;

@interface ViewController : UIViewController <SKYLINKConnectionLifeCycleDelegate>;

<strong>// 1</strong>
@end

</kbd>
#import "ViewController.h"

@interface ViewController()
@property (strong, nonatomic) SKYLINKConnection *skylinkConnection;
@property (strong, nonatomic) UIView *localRenderView;

<strong>// 2</strong>

@end

@implementation ViewController
-(void)viewDidLoad {
[super viewDidLoad];

//Creating Configuration

SKYLINKConnectionConfig *config = [SKYLINKConnectionConfig new];
 config.video = YES;
 config.audio = YES;
 config.timeout = 30;

// Creating SKYLINKConnection

 self.skylinkConnection = [[SKYLINKConnection alloc] initWithConfig:config appKey:@"YOUR-API-KEY"];
 self.skylinkConnection.lifeCycleDelegate = self;

<strong>// 3</strong>
 
// Connecting to a room

[self.skylinkConnection connectToRoomWithSecret:@"YOUR-SECRET" roomName:@"ROOM-NAME" userInfo:nil];
}

<strong>// 4</strong>

#pragma mark - SKYLINKConnectionLifeCycleDelegate
 - (void)connection:(SKYLINKConnection*)connection didRenderUserVideo:(UIView*)userVideoView {
 self.localRenderView = userVideoView;
 self.localRenderView .frame = self.view.bounds;
 self.localRenderView .autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
 self.localRenderView .translatesAutoresizingMaskIntoConstraints = YES;
 [self.view insertSubview:self.localRenderView atIndex:0];
}

@end

What we just added in the code is marked by commented numbers 1 to 4:

  1. Declare the viewController as a Temasys Skylink lifecycle delegate.
  2. Declare a UIView *localRenderView property, that we will need in the future.
  3. Set the connection’s ‘lifeCycleDelegate’ to self (the viewController).
  4. Implement the associated delegate method.

Steps 1 to 3 are pretty straightforward, if you know how to use protocols on iOS. So, let’s look at Step 4.

We are told by the connection that it did render a user video. For simplicity, we make this video (which is a UIView) take the whole screen, assign it to the property declared in step 2, and add it to self.view. Just like that, running the app on a device will end up showing the user what the front camera is filming. But wait! The video is stretched! That makes you look a little funny!

The reason for that is that the view rendering the video has a frame corresponding to the view it was added to, so it’s height/width ratio is not necessarily the same as the videos’s ideal resolution height/width ratio.

Hopefully, there is a simple way to fix that.


Correct the video stretching

Just like before, we need to implement a delegate method, which will be called when the video size changes. So, for example, when the video begins to be rendered, do the following :

  1. Declare the viewController as a ‘SKYLINKConnectionMediaDelegate’.
  2. Import AVFoundation (needed in Step 4)
  3. Set the connection’s ‘mediaDelegate’ to self (the viewController).
  4. Implement the associated delegate method, the one that will give us the size, called – (void)connection:(SKYLINKConnection*)connection didChangeVideoSize:(CGSize)videoSize videoView:(UIView*)videoView
#import <UIKit/UIKit.h>;
#import <SKYLINK/SKYLINK.h>;

@interface ViewController : UIViewController <SKYLINKConnectionMediaDelegate, SKYLINKConnectionLifeCycleDelegate>;

<strong>// 1</strong>

@end

</kbd>#import "ViewController.h"
#import <AVFoundation/AVFoundation.h>; 

<strong>// 2</strong>

@interface ViewController ()
@property (strong, nonatomic) SKYLINKConnection *skylinkConnection;
@property (strong, nonatomic) UIView *localRenderView;
@end

@implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];

// Creating configuration
SKYLINKConnectionConfig *config = [SKYLINKConnectionConfig new];
config.video = YES;
config.audio = YES;
config.timeout = 30;

// Creating SKYLINKConnection
self.skylinkConnection = [[SKYLINKConnection alloc] initWithConfig:config appKey:@"YOUR-API-KEY"];
self.skylinkConnection.lifeCycleDelegate = self;
self.skylinkConnection.mediaDelegate = self;

<strong>// 3</strong>

// Connecting to a room
[self.skylinkConnection connectToRoomWithSecret:@"YOUR-SECRET" roomName:@"ROOM-NAME" userInfo:nil];
}

#pragma mark - SKYLINKConnectionLifeCycleDelegate
- (void)connection:(SKYLINKConnection*)connection didRenderUserVideo:(UIView*)userVideoView {
self.localRenderView = userVideoView;
self.localRenderView .frame = self.view.bounds;
self.localRenderView .autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
self.localRenderView .translatesAutoresizingMaskIntoConstraints = YES;

[self.view insertSubview:self.localRenderView atIndex:0];
}

#pragma mark - SKYLINKConnectionMediaDelegate
- (void)connection:(SKYLINKConnection*)connection didChangeVideoSize:(CGSize)videoSize videoView:(UIView*)videoView

<strong>// 4</strong>
{
if (self.localRenderView &amp;&amp; (videoSize.height &gt; 0 &amp;&amp; videoSize.width &gt; 0)) {
   CGSize defaultAspectRatio = CGSizeMake(4, 3);
   CGSize aspectRatio = CGSizeEqualToSize(videoSize, CGSizeZero) ? defaultAspectRatio : videoSize;
   CGRect videoFrame = AVMakeRectWithAspectRatioInsideRect(aspectRatio, self.view.bounds);
   self.localRenderView.frame = videoFrame;
   }
}

@end

Our goal in the fresh delegate method ‘– (void)connection:(SKYLINKConnection*)connection didChangeVideoSize:(CGSize)videoSize videoView:(UIView*)videoView’ is to set the local render view’s frame to be coherent with the video resolution’s ratio.

We are given the video size in a CGSize, after some safety on this size, we use it with the AVFoundation method: ‘CGRect AVMakeRectWithAspectRatioInsideRect(CGSize aspectRatio, CGRect boundingRect)’.

This method takes a CGSize aspectRatio and a CGRect boundingRect, and returns a CGRect corresponding to the biggest rectangle possible contained in boundingRect but respecting the aspect ratio. It’s a bit like the Aspect Fit mode for UIImages.

Once we have the correct frame, we just assign it to the local refer view’s frame, and that’s it !


Bonus: Aspect fill video

We just saw how to get the video to fit a container view, using this line of code in – (void)connection:(SKYLINKConnection*)connection didChangeVideoSize:(CGSize)videoSize videoView:(UIView*)videoView

CGRect videoFrame = AVMakeRectWithAspectRatioInsideRect(aspectRatio, self.view.bounds);

Now let’s say we want to fill the container view with the video instead of fitting it, a solution for example would be to replace this line of code with:

videoView.frame = [self aspectFillRectForSize: aspectRatio containedInRect:self.view.frame];

And the implementation of this method calculating the videoView’s frame could be:

-(CGRect)aspectFillRectForSize:(CGSize)insideSize containedInRect:(CGRect)containerRect {
CGFloat maxFloat = MAX(containerRect.size.height, containerRect.size.width);
CGFloat aspectRatio = insideSize.width / insideSize.height;
CGRect frame = CGRectMake(0, 0, containerRect.size.width, containerRect.size.height);
if (insideSize.width &lt; insideSize.height) {
   frame.size.width = maxFloat;
   frame.size.height = frame.size.width / aspectRatio;
   } else {
   frame.size.height = maxFloat;
   frame.size.width = frame.size.height * aspectRatio;
   }
frame.origin.x = (containerRect.size.width - frame.size.width) / 2;
frame.origin.y = (containerRect.size.height - frame.size.height) / 2;
return frame;
}

    • Note: Find out more about Skylink for iOS, more helpful guides, and access to code samples by visiting our website. [CTA url=/products/sdks/ios/]Take Me There![/CTA]
    • Going Further: Be sure to check the sample app on Github, and in particular the TEMVideoView class and usage. It provides a way to deal with what we discussed in this article.
    • Support: If you encounter any issues or need help, visit our support portal where you can browse commonly asked questions or send us an email!

Leave a Reply

Your email address will not be published. Required fields are marked *