4:3 to 16:9 non-linear stretch tool(s) similar to superview

[sickgreg]

Hi,

I have just joined up and would like to share my non-linear stretch tool allowing you to watch in real time your full image sensor height videos without cropping the top and bottom. Many cameras do not have this feature built in.

Realtime-Superview-Stretch

https://github.com/sickgreg/Realtime...retch/releases

An HLSL (High Level Shader Language) pixel shader that works in HLSL supported players like Media Player Classic (MPC-HC) to apply a non-linear horizontal stretch (actually exponential, reversing the lens' fisheye squishy effect at the edges) in real time. This allows you to get the absolute most out of your 4:3 sensor and works great in my case for FPV drone footage.

I'm not a programmer but i "needed" this and went at it boots and all. I was inspired by the fantastic https://github.com/Niek/superview video converter tool which of course re-encodes the whole video on your CPU, great if you want to upload to YouTube, but not so great for just reviewing (in my case) the flight footage.

Please enjoy

Feel free to edit the code. It's a miracle that I even got it working using basic linear interpolation as the method of remapping pixels. The stretch can look a bit blocky because of this method. It is beyond my ability to utilize a nicer resize algorithm but probably not rocket science either. That being said, it still looks really great, especially if added to your player post resize on a large monitor.

Cheers
Greg

[nutsey]

Hi,

Thanks for sharing!

What about adding another algorithm where 16:6 picture is not stretched from the sides but is nonlinearly squeezed along its horizontal?

I've implemented it as 'Ultraview' for Firefly 8SE custom fws here: https://www.goprawn.com/forum/ambare...1984#post11984

[sickgreg]

This is what it does.... Is that the same as yours?
Nonlinearly squeezed along its horizontal and scaled to 16:9 in this case by the player. Actually there is misleading code in my .hlsl file! Left over unused variable from an earlier version:

#define PERCENTAGE_NOT_STRETCHED 0.0

It's set to zero and hence not used

[nutsey]

Mine is not upscaled but downscaled nonlinearly in perpendicular dimension.

Like this:

[sickgreg]

[sickgreg]

I've been trying to create a Realtime-Nutsey-Squash.hlsl shader but it is doing it perfectly backwards.... squishing the center instead of the top and bottom and I can't figure out the math.... Any idea??

sampler s0 : register(s0);

float4 main(float2 tex : TEXCOORD0) : COLOR
{


    tex.y = tex.y * 2.0 - 1.0;
    // convert incoming pixel coord from a value 0 to 1 to a value -1 to +1

    float distanceFromEdge = (max(  abs(tex.y), 0.0 ) * 0.5 + 0.5 );
    // pixel gets its distanceFromEdge value between 0 and 1

    tex.y = lerp( tex.y / 0.75, tex.y,    pow(distanceFromEdge,2) );
    // linear interpolate using the square of the distance from the edge

    tex.y = tex.y * 0.5 + 0.5;
    // convert outgoing pixel coord from a value -1 to +1 back to a value 0 to 1

    float4 l = tex2D(s0, tex);
    return l;    //return the pixel 
}

[nutsey]

Actually my distortion profile is for 4:3 picture linearly resized into 16:9 (common modes for Ambarella cams with 4:3 CMOS sensors) and it stretches the center to get proper geometry

[sickgreg]

Okay so the Ambarella applies lens distortion correction that you have defined somehow...? Or is the profile already pre-defined? How do you define a distortion profile and do you know if the icatch v35ma has such a feature?

Obviously the methods I posted are applied either via re-encoding (superview.exe) or in real time whilst watching on the computer in Media Player Classic with realtime-superview-stretch.hlsl

Sorry about all the questions!

Cheers
Greg

[nutsey]

Talking about Ambarella there is a distortion correction table with shift values. By default it is calculated to compensate average barrel distortion (fullframe fisheye). I completely redisigned it to improve frame geometry after 4:3->16:9 conversion.

As for iCatch, I'm going to get my V35 cam in a couple of weeks. Hope it uses an SPI flash memory that can be dumped without any soldering.