Announcement

Collapse
No announcement yet.

4:3 to 16:9 non-linear stretch tool(s) similar to superview

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • 4:3 to 16:9 non-linear stretch tool(s) similar to superview

    Hi,

    I have just joined up and would like to share my non-linear stretch tool allowing you to watch in real time your full image sensor height videos without cropping the top and bottom. Many cameras do not have this feature built in.

    Realtime-Superview-Stretch

    https://github.com/sickgreg/Realtime...retch/releases

    An HLSL (High Level Shader Language) pixel shader that works in HLSL supported players like Media Player Classic (MPC-HC) to apply a non-linear horizontal stretch (actually exponential, reversing the lens' fisheye squishy effect at the edges) in real time. This allows you to get the absolute most out of your 4:3 sensor and works great in my case for FPV drone footage.

    I'm not a programmer but i "needed" this and went at it boots and all. I was inspired by the fantastic https://github.com/Niek/superview video converter tool which of course re-encodes the whole video on your CPU, great if you want to upload to YouTube, but not so great for just reviewing (in my case) the flight footage.

    Please enjoy

    Feel free to edit the code. It's a miracle that I even got it working using basic linear interpolation as the method of remapping pixels. The stretch can look a bit blocky because of this method. It is beyond my ability to utilize a nicer resize algorithm but probably not rocket science either. That being said, it still looks really great, especially if added to your player post resize on a large monitor.

    Cheers
    Greg

  • #2
    Hi,

    Thanks for sharing!

    What about adding another algorithm where 16:6 picture is not stretched from the sides but is nonlinearly squeezed along its horizontal?

    I've implemented it as 'Ultraview' for Firefly 8SE custom fws here: https://www.goprawn.com/forum/ambare...1984#post11984
    Donate here if you want to support my efforts and this site.

    Email me if you have any offers, requests or ideas.

    Comment


    • #3
      Click image for larger version

Name:	grid_43.jpg
Views:	262
Size:	79.1 KB
ID:	16512

      Click image for larger version

Name:	grid19by9.png
Views:	240
Size:	1.24 MB
ID:	16511

      This is what it does.... Is that the same as yours?
      Nonlinearly squeezed along its horizontal and scaled to 16:9 in this case by the player. Actually there is misleading code in my .hlsl file! Left over unused variable from an earlier version:

      #define PERCENTAGE_NOT_STRETCHED 0.0

      It's set to zero and hence not used
      Attached Files

      Comment


      • #4
        Mine is not upscaled but downscaled nonlinearly in perpendicular dimension.

        Like this:
        Donate here if you want to support my efforts and this site.

        Email me if you have any offers, requests or ideas.

        Comment


        • #5
          Click image for larger version

Name:	cpOrig.png
Views:	240
Size:	620.0 KB
ID:	16531
          Click image for larger version

Name:	cpWide.png
Views:	230
Size:	812.4 KB
ID:	16532
          Click image for larger version

Name:	ampsOriginal.png
Views:	235
Size:	595.7 KB
ID:	16533
          Click image for larger version

Name:	ampsWide.png
Views:	231
Size:	832.1 KB
ID:	16534

          Comment


          • #6
            I've been trying to create a Realtime-Nutsey-Squash.hlsl shader but it is doing it perfectly backwards.... squishing the center instead of the top and bottom and I can't figure out the math.... Any idea??


            Code:
            sampler s0 : register(s0);
            
            float4 main(float2 tex : TEXCOORD0) : COLOR
            {
            
            
                tex.y = tex.y * 2.0 - 1.0;
                // convert incoming pixel coord from a value 0 to 1 to a value -1 to +1
            
                float distanceFromEdge = (max(  abs(tex.y), 0.0 ) * 0.5 + 0.5 );
                // pixel gets its distanceFromEdge value between 0 and 1
            
                tex.y = lerp( tex.y / 0.75, tex.y,    pow(distanceFromEdge,2) );
                // linear interpolate using the square of the distance from the edge
            
                tex.y = tex.y * 0.5 + 0.5;
                // convert outgoing pixel coord from a value -1 to +1 back to a value 0 to 1
            
                float4 l = tex2D(s0, tex);
                return l;    //return the pixel 
            }

            Comment


            • nutsey
              nutsey commented
              Editing a comment
              Actually my distortion profile is for 4:3 picture linearly resized into 16:9 (common modes for Ambarella cams with 4:3 CMOS sensors) and it stretches the center to get proper geometry

          • #7
            Okay so the Ambarella applies lens distortion correction that you have defined somehow...? Or is the profile already pre-defined? How do you define a distortion profile and do you know if the icatch v35ma has such a feature?

            Obviously the methods I posted are applied either via re-encoding (superview.exe) or in real time whilst watching on the computer in Media Player Classic with realtime-superview-stretch.hlsl

            Sorry about all the questions!

            Cheers
            Greg

            Comment


            • nutsey
              nutsey commented
              Editing a comment
              Talking about Ambarella there is a distortion correction table with shift values. By default it is calculated to compensate average barrel distortion (fullframe fisheye). I completely redisigned it to improve frame geometry after 4:3->16:9 conversion.

              As for iCatch, I'm going to get my V35 cam in a couple of weeks. Hope it uses an SPI flash memory that can be dumped without any soldering.

          • #8
            Hi sickgreg

            I tried to use your shader with VirtualDub2 but it does not work. Here are some shaders that work: https://github.com/Loadus/virtualdub-shaders

            To load a shader go to Options - Preferences - Display - Use Direct3D 9 - Use effect file ...

            Is it possible to modify your code in order to use it with this software?
            Donate here if you want to support my efforts and this site.

            Email me if you have any offers, requests or ideas.

            Comment


            • #9
              I tried adding
              Code:
              technique {
              pass {
              PixelShader = compile ps_3_0 main();
              }
              }
              but the video panes turned black.
              Donate here if you want to support my efforts and this site.

              Email me if you have any offers, requests or ideas.

              Comment


              • #10
                Did you get it to work in a player such as MPC-HC or MPC-BE ?

                It would be good in VirtualDub2 for processing files rather than only watching it live. I will have a look into that. I didn't even know virtual dub existed for anything other than AVI. That is cool VD2.

                I got realtime-nutsey-squash.hlsl working by the way
                Code:
                // Realtime-nutsey-squash.hlsl by sickGreg
                sampler s0 : register(s0);
                
                float4 main(float2 tex : TEXCOORD0) : COLOR
                {
                
                
                tex.y = tex.y * 2.0 - 1.0;
                // convert incoming pixel coord from a value 0 to 1 to a value -1 to +1
                
                float distanceFromEdge = (max( abs(tex.y), 0.0 ) * 0.5 + 0.5 );
                // pixel gets its distanceFromEdge value between 0 and 1
                
                tex.y = lerp( tex.y * 0.75, tex.y, distanceFromEdge*distanceFromEdge);
                // linear interpolate using the square of the distance from the edge
                
                tex.y = tex.y * 0.5 + 0.5;
                // convert outgoing pixel coord from a value -1 to +1 back to a value 0 to 1
                
                float4 l = tex2D(s0, tex);
                return l; //return the pixel
                }
                Last edited by sickgreg; 02-14-2020, 09:29 AM.

                Comment


                • #11
                  Like this!

                  realtime-superview-stretch

                  Click image for larger version

Name:	VD2_Superview_Train.gif
Views:	203
Size:	492.4 KB
ID:	17051


                  Or using realtime-nutsey-squash (Ultraview)

                  Click image for larger version

Name:	VD2_UltraView_Train.gif
Views:	183
Size:	537.3 KB
ID:	17052

                  Give me a moment and i will write up how to do it.

                  Thanks Nutsey for pointing me in the direction of VirtualDub 2.

                  Comment


                  • #12
                    Put DirectX settings back to default

                    Download http://www.virtualdub.org/downloads/vdshader-1.4.zip and extract to VirtualDub2/plugins32 folder
                    Open 32-bit version of VirtualDub2 and open your video file
                    Click Video/Filter and add vdshader
                    Copy and paste the code into the vdshader window
                    Code:
                    //realtime-superview-stretch.hlsl by sickGreg
                    sampler s0 : register(s0);
                    
                    float4 main(float2 tex : TEXCOORD0) : COLOR
                    {
                    
                    // convert incoming pixel coord from a value 0 to 1 to a value -1 to +1
                    tex.x = tex.x * 2.0 - 1.0;
                    
                    // give pixel its distanceFromEdge value between 0 and 1
                    float distanceFromEdge = max( abs(tex.x ), 0.0 ) * 0.5 + 0.5 ;
                    
                    // linear interpolate using the square of the distance from the edge
                    tex.x = lerp( tex.x / 0.75, tex.x, pow(distanceFromEdge,2) );
                    
                    // convert outgoing pixel coord from a value -1 to +1 back to a value 0 to 1
                    tex.x = tex.x * 0.5 + 0.5;
                    
                    //return the pixel
                    float4 l = tex2D(s0, tex);
                    return l;
                    }
                    Save As "superview-stretch.fx"
                    Close the vdshader window and the shader should now be applied to the video preview
                    You'll also need to add the resize filter and resize the horizontal to 133.333333333%
                    I figure the results should be better resizing first and then non-linear stretching
                    Last edited by sickgreg; 02-14-2020, 09:37 AM.

                    Comment


                    • nutsey
                      nutsey commented
                      Editing a comment
                      It works! Thank you!
                  Working...
                  X