Announcement

Collapse
No announcement yet.

Novatek command line tools

Collapse
This is a sticky topic.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Please don't use bfc -p command for now. There seems to be some issue in some rare conditions.
    I'm almost finished with rewriting bfc completely (except the lz77 library) to handle all types of binaries better (still no multipartition support but I'm also working on a firmware split application).
    This includes auto-detection of partcomp offsets

    Comment


    • #17
      Perepandel the file you provided seems to work fine with my latest build of bfc (which is not public yet) but doesnt work with the latest public bfc. So I'm going to share the pre-release version: http://dc.p-mc.eu/ntk_tools/bfc4ntk_pre.exe
      1. Use this tool to unpack the firmware. (Using the command -p: bfc4ntk -p FWDV180.bin FWDV180.rbn
      2. Modify the .rbn file using a hex editor or other tools like NtkMPE
      3. -- We are not creating a partcomp binary again, instead we make it fullcomp: --
      3.1 Calculate the checksum of the .rbn file using ntkcalc (ntkcalc -cw FWDV180.rbn)
      3.2 Repack the binary using bfc4ntk: bfc4ntk -c FWDV180.rbn mod.bin
      3.3 Calculate the checksum of the modified packed binary: ntkcalc -cw mod.bin

      This should result in a valid flashable firmware binary.

      I'm not sure why you got the invalid checksum magic error, I wasn't able to reproduce that since (public)bfc is unable to unpack the file.

      Comment


      • #18
        Hi [email protected]!

        Wow, I'm really impressed by your work. I already read your messages yesterday but was so tired and sleepy after work that I couldn't even download the tools, which is what I'm trying to do now.

        Will keep you informed about my progress. And again, many, many thanks and cheers for your work. It's amazing!!

        Also, I don't know if this is useful for anything, but I discovered that this machine is sensible to the "engmode". I mean... I tried to insert an SD with just an empty "engmode" (without extension) file on it and when it turns on, the screen is put in red and some error messages appear, obviously because it's looking for some files but it doesn't find anything:

        "jpg size is too big or too"

        "update logo2 failed"

        "cfg file error: Open"

        Then it turns off again.

        I wish I could do a backup of my original firmware before attempting to flash the machine...

        EDIT:

        I just found this... http://git.p-mc.eu/Tobi/ntk_sdk/blob...IConfig.c#L869

        Last edited by Perepandel; 08-09-2017, 09:35 PM.

        Comment


        • #19
          Actually you should be safe without backing it up since you still have an original firmware (update) binary.
          Novatek is almost unbrickable if you don't flash the bootloader. The bootloader is a separate binary which is not included in the firmware binary.

          The bootstrapper/bootloader is the first stage of software after you power up the device and this piece of software flashes firmware updates. Even after flasing a completely corrupt firmware file the bootloader is still fully functional. All you have to do is putting a valid firmware binary onto the sd card and the bootloader will flash it.

          Since you have the original firmware update you should be safe.

          I hope this makes sense

          Comment


          • #20
            By the way, I started "playing" and got a bit confused:

            - What I did on my previous attempts was opening the firmware in bnGui BETA 0.3 ("Load firmware" button). I pressed "Unpack" and that way is how I got an .RBN file.

            - Using this same bnGUI BETA 0.3 I tried "Compress" after clicking on "Open" in the "Compress" section and opening the .RBN file on the previous step. That's what lead the the "NATIVE ERROR - Checksum magic invalid (432e)" error.

            Now reading your instructions you say:

            "1. Use this tool to unpack the firmware. (Using the command -p: bfc4ntk -p FWDV180.bin FWDV180.rbn"

            But when I rung bfc4ntk_pre, it says that the "-p" parameter does COMPRESS the file.

            So what are we doing in the end? Is my original .bin file a compressed or an uncompressed image? Was there are typo in your message or is the type in bfc4ntk_pre help?

            The .rbn file I'm getting now it's pretty close in size to my original .bin file (2295 kB for the .rbn vs 2352 for the original .bin), and they look "similar" when inspected using a hex editor, but then, the size of the resulting .bin after repacking it with bfc4ntk -c is smaller: 1962 kB. Shouldn't it be the same size if I haven't done any modification? The resulting .bin starts with a "BCL1" string, instead of the series of dots I see in my original .bin file...

            Quite strange, also because the .rbn file I got with the bnGui method I described above starts neither with any of the "strings" I commented (with "rMov#"), and has a lot of readable strings.

            I'll try to follow the steps again to see if I've missed something...

            Comment


            • #21
              Originally posted by Perepandel View Post
              By the way, I started "playing" and got a bit confused:

              - What I did on my previous attempts was opening the firmware in bnGui BETA 0.3 ("Load firmware" button). I pressed "Unpack" and that way is how I got an .RBN file.

              - Using this same bnGUI BETA 0.3 I tried "Compress" after clicking on "Open" in the "Compress" section and opening the .RBN file on the previous step. That's what lead the the "NATIVE ERROR - Checksum magic invalid (432e)" error.

              Now reading your instructions you say:

              "1. Use this tool to unpack the firmware. (Using the command -p: bfc4ntk -p FWDV180.bin FWDV180.rbn"

              But when I rung bfc4ntk_pre, it says that the "-p" parameter does COMPRESS the file.

              So what are we doing in the end? Is my original .bin file a compressed or an uncompressed image? Was there are typo in your message or is the type in bfc4ntk_pre help?

              The .rbn file I'm getting now it's pretty close in size to my original .bin file (2295 kB for the .rbn vs 2352 for the original .bin), and they look "similar" when inspected using a hex editor, but then, the size of the resulting .bin after repacking it with bfc4ntk -c is smaller: 1962 kB. Shouldn't it be the same size if I haven't done any modification? The resulting .bin starts with a "BCL1" string, instead of the series of dots I see in my original .bin file...

              Quite strange, also because the .rbn file I got with the bnGui method I described above starts neither with any of the "strings" I commented (with "rMov#"), and has a lot of readable strings.

              I'll try to follow the steps again to see if I've missed something...
              Oops sorry, -p actually means partcomp, so yeah this command compresses the file. You need the same syntax using -x command.
              The other instructions should be ok

              The resulting file indeed starts with BCL1 since we are creating a fullcomp binary (which means fully compressed instead of partcomp which means raw binary portion + compressed binary portion at address X) The bootloader will be able to detect the compression type. I don't know why they even introduced the partcomp type. Maybe it increases the bootup process a little bit since only a smaller part needs to be decompressed. But then again decompression with lz77 is super fast anyways, since its based on pointers
              Last edited by [email protected]; 08-09-2017, 10:26 PM.

              Comment


              • #22
                Quick update as it's 2:00 am in the morning and I'm getting up at 7 am :P

                I tried your recipe but it doesn't work, [email protected] The machine does nothing when I put the modified firmware inside a blank SD card and turn it on. It does nothing. Nada. Niente. It even refuses to turn on. Not a single blink on the "on" led which, instead, does a series of blinkings in different speeds and durations when flashing with the official updated.

                I even have to disconnect the machine from the power supply in order to be able to turn it on again.

                It looks to me like this machine doesn't like fullcomp binaries. If I'm not wrong, I still have a change of manually generating a partcomp image by doing some replacements starting from some address on the original firmware, true? Anyway that will be the next day. I'll also give this recipe another try because I'm not sure about having done it right. Theoretically I just modified the bitrate, fps and resolution to the video mode on the list I know it's being used for the output video files of my machine.

                Here it goes a Mega link to my modded attempt of firmware image, which is about 300 kb smaller compared to the official firmware. I also tried changing the file name from "mod.bin" to "FWDV180N.bin". Maybe it's important to keep the filenames in every step. Instead, I just renamed the file after generating the supposedly working firmware. Who knows...

                https://mega.nz/#!tb5wUBzD!GgvbO1RzH...BWFs0gHLdaETKU

                EDIT:

                I just tried the steps again and freshly after a very long and repairing sleep tonight and I can confirm it doesn't work. As I said previously, the machine doesn't even seem to recognize the file as a valid firmware and none of the blinking on the led happens.

                Now I'm going to try Michi_65 recipe on post #11 on this same thread
                Last edited by Perepandel; 08-12-2017, 03:59 PM.

                Comment


                • #23
                  Well, Michi_65's recipe didn't work, either

                  Following his method I got a file the same size as the original firmware, as expected, but the machine won't attempt to flash it, either.

                  There a couple of suspicious issues here:

                  - 1st of all, are you sure the machine would accept files with the BCL1 header, considering the original firmware image starts with a series of dots??

                  - Could this be a naming issue? I mean... I cannot put an arbitrary name for the file; ie, if I name the file as "test.bin", the machine will happily ignore it, starting in working mode. If I name the file the same as my original firmware, that is, FWDV180N.bin, it does nothing when trying to turn it on, which I assumed was due to the file not being in the proper format, but... Could it be that, as it notices (by name) it's the same firmware image, it simple ignores it?

                  I have no clue how the machine identifies valid firmware file names. I tried to rename it to FWDV181N.bin, but it also got ignored, starting in normal mode.

                  Now I've really come to a dead end . I need of your savyy knowledge, guys. The bitrate on the output video files is just about 8 mbps; you can get an idea on how awful it looks... :/

                  Comment


                  • #24
                    Originally posted by Perepandel View Post
                    Well, Michi_65's recipe didn't work, either

                    Following his method I got a file the same size as the original firmware, as expected, but the machine won't attempt to flash it, either.

                    There a couple of suspicious issues here:

                    - 1st of all, are you sure the machine would accept files with the BCL1 header, considering the original firmware image starts with a series of dots??

                    - Could this be a naming issue? I mean... I cannot put an arbitrary name for the file; ie, if I name the file as "test.bin", the machine will happily ignore it, starting in working mode. If I name the file the same as my original firmware, that is, FWDV180N.bin, it does nothing when trying to turn it on, which I assumed was due to the file not being in the proper format, but... Could it be that, as it notices (by name) it's the same firmware image, it simple ignores it?

                    I have no clue how the machine identifies valid firmware file names. I tried to rename it to FWDV181N.bin, but it also got ignored, starting in normal mode.

                    Now I've really come to a dead end . I need of your savyy knowledge, guys. The bitrate on the output video files is just about 8 mbps; you can get an idea on how awful it looks... :/
                    Sorry was busy until now. If it does nothing (not even boot) with the firmware placed in the sdcard root (it's name has to be FWDV180N.bin in your case) thats because the bootloader indeed refuses to flash the firmware because it cant be verified.
                    Regarding naming issues: Most of the time the filename of the update binary is always the same (no increasing number or similar things)

                    As you guesses it might not like the fullcomp format. Thats a really strange behavior but could be possible. So we are going to try partcomp for now with the steps below:
                    1. Decompress the original file: bfc4ntk -x FWDV180N.bin unpacked.bin
                    2. Modify the raw binary
                    3. Compress the modified binary: bfc4ntk -p unpacked.bin mod.bin
                    (-p should automatically detect the offset address)
                    4. Calculate its checksum: ntkcalc -cw mod.bin
                    5. Put the mod.bin file into the sdcard root and rename it to FWDV180N.bin
                    6. We'll see if it works this time

                    Information regarding bootloader LED flashing states:
                    Slow flashing LED: erasing memory blocks
                    Fast flashing LED: writing new data to memory blocks

                    Comment


                    • #25
                      IT WORKS!!

                      It's flashing it!

                      More info later!

                      EDIT 1:

                      I'ts flashed but I ruined the machine xD. It "turns on" because the scanner light (remember, this is a film scanner) turns on, but nothing is displayed on the screen.

                      I'm going to try to reflash with the original firmware and do further testing... xD

                      EDIT 2:

                      Well, no worries - no harm done. The machine can be flashed back with its original firmware without damage

                      But prior to that I did a simple test: followed [email protected] steps but on point 2 I didn't modify anything - I simply decompressed the original file, than compressed it back again and calculated the checksum. It was properly flashed but, again, I got a non-functional machine!

                      Then, I compared the generated files to the original .bin image. Appart from three single characters difference @offset 62, and three other at offset 000AB004, at the bottom of the file there are extra offsets going from 0x0024BC10 to 0x0024BC64.

                      [email protected], I don't know if you are taking a look at this, but I can upload the generated, non-working .bin image. Althought as I sent it you previously there is no need for so. Please let me know in case you want me to upload it.

                      These are the "extra bits" at the bottom of the generated file:

                      D040C9D8C4082B7199D8C418C419D990299029DB204B2049DE 408E4089D81C81081C8109D8390208390209D868D1F868D1F9 D868D1F868D1F9D868D1F868D1F9D868D1F868D1F9D868D1F8 68D1F9D85FD2585FD25000000
                      Last edited by Perepandel; 08-13-2017, 01:46 PM.

                      Comment


                      • #26
                        By how much did you increase the bitrate?

                        Comment


                        • Perepandel
                          Perepandel commented
                          Editing a comment
                          Hi [email protected]! It seems my previous edit got later than this message. I increased it by 10x, but that doesn't matter because as you can see in my #Edit 2, I took the same test without modifications - no bitrate increase, nothing. Just unpacking and regenerating the file. And the result was a non-functional machine. So the problem must be located somewhere else...

                      • #27
                        Perepandel please provide the unmodified recompressed file, things are getting strange now

                        Comment


                        • Perepandel
                          Perepandel commented
                          Editing a comment
                          Of course [email protected]! Here it goes:

                          https://mega.nz/#!ofgRVTqT!TKM4HBSe4...0ltWDZghoPz-Lk

                          I didn't had much hope in the beginning about getting this to work - but now that I saw the machine flashing the modified firmware, and seeing your dedication, I'm starting to think that some success is possible!

                          In any case, I'm learning a lot and I find this very instructive and even getting fun from this. I've always wished I could do these hackings myself. I also feel ashamed because I didn't notice I could have tried those commands by myself without the need for waiting to your explanations - everything was on the utilities help, but it's just that I didn't realized what some parts of the process really meant and that all. I think I'm getting old!
                          Last edited by Perepandel; 08-13-2017, 11:37 PM. Reason: typo

                        • Tobi@s
                          [email protected] commented
                          Editing a comment
                          You've actually discovered a bug in bfc4ntk @Perepandel
                          The packed portion is missing the compressed size inside the bcl header at address 0x000ab00c. I will fix bfc4ntk asap.

                      • #28
                        Fixed a bug in bfc4ntk pre-release I've found thanks to Perepandel
                        Please redownload v3.1 from http://dc.p-mc.eu/ntk_tools/bfc4ntk.exe and follow the partcomp instructions once again

                        Comment


                        • #29
                          Hey [email protected]! Thanks for your effort!

                          I just followed your instructions but got the same result... The machine doesn't turn on (appart from the scanner light), and I still see the difference between the original firm and the repacked one (without modifications), specially the extra offsets at the bottom.

                          Are you sure your modifications made it into the v3.1 that's available for download?

                          Edit:

                          Here it goes the link to my "regenerated" firm:

                          https://mega.nz/#!QLI3GbCB!6PUVqsxph...6--G2aro-zkAwA

                          I see subtle differences in two offsets betwen the last one I uploaded and this one, but the final "extra" part compared to the original one is what I find more suspicious...
                          Last edited by Perepandel; 08-14-2017, 11:32 AM.

                          Comment


                          • #30
                            Yes you've used the latest version Perepandel and I've got good and bad news for you:
                            Bad news first, I have no idea whats going on there. The extra bytes at the end of the file are generated by lz77 library I'm using. To be honest I don't have very much knowledge regarding compression algorithm so debugging the extra bytes for now is most likely not going to happen too soon. There are some constants in the compression library that result in different output sizes and I think that _maybe_ novatek uses a slightly different configuration but I have no idea which one.
                            Good news: Well partially: I can reproduce your issue. I also have a partcomp camera lying around which behaves like your video converter. It flashes the partcomp firmware I created with bfc but doesnt boot. Good thing is that I have access to the debug console of the camera so I might get an idea of whats wrong. Will update you later if I find anything interesting

                            Comment

                            Working...
                            X