Programming the Guide Lights :-)

Discussion in 'KOMPLETE KONTROL SERIES' started by Anykey, Sep 21, 2017.

  1. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    I tried to implement your function what "nearly" works, but image.bits() seems always to return 32bit argb data instead of 16bit rgb - even if the convertToFormat function was called before. On a 256x256 image, image.byteCount() and the size of your swappedData array are 262144 - that is 4 bytes per pixel.

    When I treat the image as a 512x512 image, it is displayed in a raster of wrong colors in the double size.
     
  2. j0tt

    j0tt New Member

    Messages:
    12
    Not sure what's your problem here...
    Code:
    #include <QApplication>
    #include <QImage>
    #include <QDebug>
    
    int main(int argc, char *argv[])
    {
        QApplication a(argc, argv);
        QImage imageARGB32(480, 270, QImage::Format_ARGB32);
        qDebug() << imageARGB32;
        QImage imageRGB16 = imageARGB32.convertToFormat(QImage::Format_RGB16);
        qDebug() << imageRGB16;
        return 0;
    }
    
    Outputs
    Code:
    QImage(QSize(480, 270),format=5,depth=32,devicePixelRatio=1,bytesPerLine=1920,sizeInBytes=518400)
    QImage(QSize(480, 270),format=7,depth=16,devicePixelRatio=1,bytesPerLine=960,sizeInBytes=259200)
    
    Keep in mind QImage::convertToFormat returns a COPY of the image and is const otherwise...
     
  3. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    Got it. Problem was that I had to call

    Code:
    image = image.convertToFormat(QImage::Format_RGB16);
    
    and not just

    Code:
    image.convertToFormat(QImage::Format_RGB16);
    
    I changed the byteswap-Function a little bit because the listed function is MSVCC-specific. For a productive software, it seems to be required to deal with the compiler flags that MSVCC and GCC set.

    The good news is that the picture appears on the displays in the right colors now.

    Bad news is that when a 256x256 pixel image should be shown, I get an image in this size what contains two times the picture in 128x128, on beside the other and garbage in 256x128 below. I have to examine all steps of the data conversion too see what is wrong.
     
  4. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    So... finally I got it working.

    I attached an updated crappy bitmap test tool which now allows you to choose an image file to transmit it to the selected display.

    One restriction yet is that the width and the height of the image both need to be dividable by 2. (256x256 works, 255x255 produces garbage). Yes, it is the else if handling if there is one remaining pixel but I am too lazy yet to solve this.

    Additionally please be sure that the selected image does not exceed the screen dimensions. If you try this, the Komplete Kontrol does not accept other images anymore until you do a power cycling.

    By the way, the byte swapping function is finally not required and commented out.

    The code should compile if QT4 and libusb are installed. Linux binary is included as well as a sample image for testing in different sizes.

    Next step is to integrate the functions into my qKontrol software to enhance it with screen data. But now I am on holidays for two weeks.
     

    Attached Files:

  5. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    I found out that you may use libusb to send images to the screens but you have to use hidapi to configure the other keyboard parameters because the Linux HID driver claims the HID endpoint of the device what makes a direct communication through libusb impossible.

    So you need to open the device twice, endpoint 02 through HIDAPI and endpoint 03 through LIBUSB.
     
  6. moss

    moss NI Product Owner

    Messages:
    170
    Yes, that's how I already do it with my Kontrol 1 implementation for Bitwig.

    Some things I found out during your absence:
    - The Maschine Mk3 spec posted above is totally identical to Kontrol 2!
    - ... except the Endpoint
    - ... and it misses command 0x02, but you alredady found out about that.
    - But the code examples above (based on that spec) are not correct (e.g. they use 2 bytes for the command); look at the example code in the original repo instead
    - Using command 0x02 to send an image is VERY slow, use 0x00 instead. But also with 0x00 you should send the whole image at once, since single commands also take some time (I tried to send the lines separately before and could watch them being drawn).

    I could not open the enpoint so far on Windows or Mac.
     
    Last edited: Sep 9, 2018
  7. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    It is possible that the command 00 on the Komplete MK2 can be used to transmit one following Machine MK3 compatible command.

    For example, 00 40 ... when the image is finished. In the MK3 protocol documentation the same begins just with 40.

    But the Komplete Kontrol software itself uses to command 00 only to introduce the footer data, nowhere else. It always transmits only restricted amounts of pixel, even on large product logos.

    The header data is exactly the same on Maschine MK3 and Komplete Kontrol MK2. Regarding the pixel data, each command 0xn transmits n² RGB565 pixels (tested yet, even works with 0xFF=510 pixels at once), except if the command is 00 and 02.
     
    Last edited: Sep 9, 2018
  8. moss

    moss NI Product Owner

    Messages:
    170
    Ah, sorry I meant 0x01 not 0x00 (I corrected it in the message above).
     
  9. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    0x01 is to transmit exactly two pixels.
     
  10. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
  11. j0tt

    j0tt New Member

    Messages:
    12
    Yes, the protocol is probably the same for Machine Mk3 and Kontrol 2 - unfortunately I don't have a Machine yet :(.

    But as mentioned in my post above (and as there is no official spec to confirm) my interpretation is a bit different.

    My understanding is that a 16-bit word for the command and a 16-bit word as "argument" is used (followed by 32-bits of data blocks). This makes sense from an embedded system view too as we keep a 32-bit alignment with each command. Also the lower half word is not used for anything in the commands as far as I see anyway. It also fits in nicely with the "COMMAND_START_OF_DATA" (0x8400) command that seems a bit convoluted in Drachenkaetzchens analysis to me.

    With the "COMMAND_TRANSMIT_PIXEL" (0x0000) and using my code I can easily push acceptable framerates of full image updates to the device - so the implementation at least works :) . I also made a full parser using my own "spec" and I could easily mirror the display with that including incremental updates.

    As for opening the endpoint, it's a bit trickier on Windows (haven't looked at it on my Mac yet) as the NIHardwareServices holds the handle for the proper USB device, also libusb is a bit fiddly without messing with custom drivers.

    So the easiest way I found is to talk to the NI Driver directly using the proper IOCTL code. So something like this (where "data" is the frame generated with something similar to the above implementation). Maybe something like this works on the Mac too... we have to try...
    Code:
    ...
    // This could be different - you should actually query this information...
    #define wszDrive L"\\??\\usb#vid_17cc&pid_1610&mi_03#9&14b62a56&0&0003#{c404974a-c9c1-4477-a9e1-9f7b8e0e4bc9}"
    // This is the magic ioctl number gained by capturing traffic...
    #define IOCTL_NI_DRIVER 2156472448
    ...
    hDevice = CreateFileW((LPWSTR)wszDrive, 0, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);
    ...
    DWORD dummy = 0;
    BOOL result = DeviceIoControl(hDevice, IOCTL_NI_DRIVER, (LPVOID)data.constData(), data.length(), NULL, 0, &dummy, (LPOVERLAPPED)NULL);
    ...
    CloseHandle(hDevice);
    
    Maybe I should write this up more formally and make a pull request to Drachenkaetzchens "spec"......
     
  12. moss

    moss NI Product Owner

    Messages:
    170
    Now, I confused myself, should have looked at my code again before answering :) Sorry.
    0x00 is the command to transmit pixels. The code to end the transmission is 0x40!
     
  13. moss

    moss NI Product Owner

    Messages:
    170
    No, it is definitively 1 byte. Look atDrachenkaetzchens specification again. It is a bit difficult to understand, it also took me some time to wrap my head around it. The transmit pixel command is:
    - 0x00 for the command
    - 3 byte for the number of pixels (strangely divided by 2), MSB first
    - The pixels encoded as 565 color.

    My current implementation is here: https://github.com/git-moss/DrivenB.../mkii/controller/Kontrol2DisplayProtocol.java

    Hmm, I also tried without the service. My problem is that I get a "descriptor couldnt be read" error on Windows.

    Yikes, not sure if I want to go there but thanks :)
     
  14. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    That means, there are two working ways to transmit pixels and if the 0x00 command is followed by a 0x40, the second byte is not part of the number of pixel but signalizes the end of transmission?
     
  15. j0tt

    j0tt New Member

    Messages:
    12
    I humbly think Drachenkaetzchen is wrong in this regards and it doesn't match the Wireshark dumps I have.
    Also this logic doesn't match what you would expect from an embedded system.

    Finally looking at your code...
    Code:
    buffer.put ((byte) (l >> 16));
    buffer.putShort ((short) (l & 0x0000FFFF));
    
    The first line always yields 0 for a short and the second line just matches what I am claiming... so just keep it simple :p

    This also holds for
    Code:
    buffer.put (COMMAND_SKIP_PIXEL);
    
    As 0x02 in my dumps is always followed by a 0x00 and then a short (big endian of course) with the number of pixels to skip. So like..
    Code:
    buffer.put (COMMAND_SKIP_PIXEL);
    buffer.put ((byte) 0x00); // Or better make COMMAND_SKIP_PIXEL 16-bit ..hehe
    buffer.putShort ((short)skip);
    
    Well basically it's just talking to a Windows driver but I am not sure if there is a Java way that works without JNI or Win32 API bindings....but this way it works without stopping the NIHardwareService (minus the interference when the Kontrol software tries to update the screen..)
     
  16. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    It seems to depend on the command if it has a size of one byte or two bytes.

    00 00 to draw pixels
    02 00 to skip pixels
    00 40 to finish the image

    but:

    01 short short to draw exactly two pixels
    03 short short short short short short to draw exactly six pixels
    and so on

    the second way seems to be mostly used by the Komplete Kontrol software
     
  17. j0tt

    j0tt New Member

    Messages:
    12
    Are you sure about this and could provide a full dump of an image update call with it?
    As for me 0x03 0x00 0x00 0x00 seems to be a "blit" operation (so dumping the current backbuffer to the screen) and that without any data, while 0x01 0x00 [ 16-bit word of repetitions in BE (n) ] [ 565-color1 ] [ 565-color2 ] makes the two given colors repeated n-times.....

    (also its 0x40 0x00 for me all the time..)
     
  18. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    see post 32 for an example

    my code (partially based on your earlier example) works

    Code:
    void qkontrolWindow::drawImage(uint8_t screen, QPixmap *pixmap)
    {
      QByteArray tux;
    
      QImage image = pixmap->toImage();
      image = image.convertToFormat(QImage::Format_RGB16);
      ushort *swappedData = reinterpret_cast<ushort *>(image.bits());
    
      tux.append(QByteArray::fromHex("8400"));
      tux.append(screen);
      tux.append(QByteArray::fromHex("600000000000000000"));
      tux.append(QByteArray::fromHex(QByteArray::number(image.width(),16).rightJustified(4,'0')));
      tux.append(QByteArray::fromHex(QByteArray::number(image.height(),16).rightJustified(4,'0')));
      bool finished = false;
      unsigned int i=0;
      unsigned int length = image.byteCount()/2;
    
      while(finished==false)
      {
      if(length-i != 0)
      tux.append(QByteArray::fromHex("02000000000000"));
      if(length-i >= 510)
      {
      tux.append(QByteArray::fromHex("ff"));
      for(unsigned int j=0;j<510;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=510;
      }
      else if(length-i >= 256)
      {
      tux.append(QByteArray::fromHex("80"));
      for(unsigned int j=0;j<256;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=256;
      }
      else if(length-i >= 128)
      {
      tux.append(QByteArray::fromHex("40"));
      for(unsigned int j=0;j<128;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=128;
      }
      else if(length-i >= 64)
      {
      tux.append(QByteArray::fromHex("20"));
      for(unsigned int j=0;j<64;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=64;
      }
      else if(length-i >= 32)
      {
      tux.append(QByteArray::fromHex("10"));
      for(unsigned int j=0;j<32;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=32;
      }
      else if(length-i >= 16)
      {
      tux.append(QByteArray::fromHex("08"));
      for(unsigned int j=0;j<16;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=16;
      }
      else if(length-i >= 8)
      {
      tux.append(QByteArray::fromHex("04"));
      for(unsigned int j=0;j<8;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=8;
      }
      else if(length-i >= 6)
      {
      tux.append(QByteArray::fromHex("03"));
      for(unsigned int j=0;j<6;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=6;
      }
      else if(length-i >= 2)
      {
      tux.append(QByteArray::fromHex("01"));
      for(unsigned int j=0;j<2;j++)
      tux.append(QByteArray::fromHex(QByteArray::number(swappedData[i+j],16).rightJustified(4,'0')));
      i+=2;
      }
      else
      {
      tux.append(QByteArray::fromHex("02000000030000"));
      tux.append(QByteArray::fromHex("0040000000"));
      finished = true;
      }
    
      }
    
    This way, a maximum of 510 pixels is beeing transmitted at one command call but that is fast enough to send two full-sized images to both displays in less than one second.

    maybe 00 03 could be for a blit operation and 00 01 for repeating pixels.

    But regarding the 01 commands there are some curiosities. It is sometimes followed by two pixels and sometimes followed by more data.
     
    Last edited: Sep 9, 2018
  19. j0tt

    j0tt New Member

    Messages:
    12
    Erm, beside that I would uphold the KISS principle in general... you seem to split the update in too many chunks ... this may lead to your confusion about the protocol...

    Basically you do a "hey here come some pixels we have 256 blocks coming!", (256 blocks are coming) , "hey here come some pixels we have 256 blocks coming"........ and so on until "hey here come some pixels we have 3 blocks coming" (3 blocks are coming) instead of just saying "hey here come some pixels we have 63920 blocks coming" (63920 blocks are coming)....

    Edit:
    Also you start each of those commands with a 02000000000000xx so you basically say "hey skip 0 pixels and here comes xx blocks of data (command 0x00 0x00)" each time too...
    and in the end
    Code:
    tux.append(QByteArray::fromHex("02000000030000"));
    tux.append(QByteArray::fromHex("0040000000"));
    
    So you again say "hey skip no pixels" and then 0x03 0x00 0x00 0x00 to blit and finally a 0x40 0x00 0x00 0x00....
     
    Last edited: Sep 9, 2018
  20. GoaSkin

    GoaSkin NI Product Owner

    Messages:
    98
    Maybe it goes better but that is what I got from wireshark. Except that the Komplete Kontrol software always transmit much smaller chunks - never saw more than 12 pixels at once there.