Skip to content

ofPixels - artifacts / crashes with .resize() #6226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
johanjohan opened this issue Feb 2, 2019 · 56 comments
Open

ofPixels - artifacts / crashes with .resize() #6226

johanjohan opened this issue Feb 2, 2019 · 56 comments

Comments

@johanjohan
Copy link

Since ofPixels.cpp has not been updated for over a year in the master branch, i therefore assume the following issue has not been fixed. i am using of 0.10.1 with win10 vs2017.

I am resizing images/pixels with the following function.
That leads to issues:

  • OF_INTERPOLATE_NEAREST_NEIGHBOR crashes on large sizes
  • OF_INTERPOLATE_BILINEAR: produces grey only pixels. well yes, "not implemented yet" in ofPixels
  • OF_INTERPOLATE_BICUBIC produces some singular green pixel/blue artifacts

calling the same function using image.resize() produces best results

question: i assume that i may use pixelsRes = _pixels; to create a full twin copy, right?
if yes, then there is the described issue in the pixel resizeTo() function.

CODE:

#include "ofApp.h"

/*
	this function aims to either
		- crop the image to the largest possible dimensions respecting _aspect 
			then scales the longest side to _longestSideLengthPx
		- stretch it respecting _aspect
			then scales the longest side to _longestSideLengthPx
*/

static void resize(
	ofPixels	 &_pixels,
	const float	 &_longestSideLengthPx,
	const ofVec2f	 &_aspect,
	const bool	 &_bStretch, // otherwise crop
	const ofInterpolationMethod	&_interp	// OF_INTERPOLATE_NEAREST_NEIGHBOR OF_INTERPOLATE_BILINEAR
) {
	ofScaleMode _scaleMode = _bStretch ? OF_SCALEMODE_FILL : OF_SCALEMODE_FIT;
	ofRectangle rectPixels(0, 0, _pixels.getWidth(), _pixels.getHeight());
	ofRectangle rectAspect(0, 0, _aspect.x, _aspect.y);
	rectAspect.scaleTo(rectPixels, _scaleMode);

	ofPixels pixelsRes;
	if (_scaleMode == OF_SCALEMODE_FIT) { // crop
		_pixels.cropTo(pixelsRes, rectAspect.x, rectAspect.y, rectAspect.width, rectAspect.height);
	}
	else { // stretch
		pixelsRes = _pixels;	
		
		// Q: is this a complete copy with new identical pixels?

		// looks like it does copy all params plus the pixels:
		//template<typename PixelType>
		//void ofPixels_<PixelType>::copyFrom(const ofPixels_<PixelType> & mom) {
		//	if (mom.isAllocated()) {
		//		allocate(mom.getWidth(), mom.getHeight(), mom.getPixelFormat());
		//		memcpy(pixels, mom.getData(), getTotalBytes());
		//	}
		//}
	}

	ofRectangle rectSize;
	if (rectAspect.width > rectAspect.height) {
		assert(rectAspect.width > 0);
		rectSize.set(0, 0, _longestSideLengthPx, _longestSideLengthPx / rectAspect.width * rectAspect.height);
	}
	else { 
		assert(rectAspect.height > 0);
		rectSize.set(0, 0, _longestSideLengthPx / rectAspect.height * rectAspect.width, _longestSideLengthPx);
	}

	ofLogNotice(__FUNCTION__) << "scaling _pixels: rectSize: " << rectSize;
	ofLogNotice(__FUNCTION__) << "_aspect   : " << _aspect;
	ofLogNotice(__FUNCTION__) << "_scaleMode: " << _scaleMode;
	ofLogNotice(__FUNCTION__) << "_interp   : " << _interp;
	ofLogNotice(__FUNCTION__) << "rectPixels: " << rectPixels;
	ofLogNotice(__FUNCTION__) << "rectAspect: " << rectAspect;

#if 1
	// ISSUE is right here

	// OF_INTERPOLATE_NEAREST_NEIGHBOR crashes on large sizes
	// OF_INTERPOLATE_BILINEAR : produces grey only pixels.well yes, "not implemented yet" in ofPixels
	// OF_INTERPOLATE_BICUBIC produces some  singular green pixel / blue artifacts
	// calling above function via image.resize() produces best results
	bool b = pixelsRes.resize(rectSize.width, rectSize.height, _interp); // THIS SEEMS BUGGY  in 0.10.1
	assert(b);
	_pixels = pixelsRes;
#else
	// all ok - done with freeimage functions
	ofImage image(pixelsRes);
	image.resize(rectSize.width, rectSize.height); // freeimage
	_pixels.setFromPixels(
		image.getPixels().getData(),
		image.getWidth(),
		image.getHeight(),
		image.getImageType()
	);
#endif
}
//--------------------------------------------------------------
void ofApp::setup() {

	ofSetLogLevel(OF_LOG_NOTICE);

	ofImage image;
	int longestSide = 15000; // somehow works with small size like 1000, crashes on large sizes
	ofInterpolationMethod interp;

	for (int i = 1; i <= 3; i++)
	{
		ofLogNotice(__FUNCTION__) << i;
		switch (i)
		{
		case 1: interp = OF_INTERPOLATE_NEAREST_NEIGHBOR; break;
		case 2: interp = OF_INTERPOLATE_BILINEAR; break;
		case 3: interp = OF_INTERPOLATE_BICUBIC; break;
		default:
			std::exit(1);
			break;
		}

		bool b = image.load("face.jpg");
		assert(b);

		bool bStretch = true;
		image.load("face.jpg");
		resize(image.getPixels(), longestSide, ofVec2f(16, 9), bStretch, interp);
		// how does the image know that the pixels are scaled now?
		image.setFromPixels(image.getPixels()); // ???
		image.save("face_169_stretch_" + ofToString(i) + ".jpg");

		image.load("face.jpg");
		resize(image.getPixels(), longestSide, ofVec2f(9, 16), bStretch, interp);
		image.setFromPixels(image.getPixels()); // ???
		image.save("face_916_stretch_" + ofToString(i) + ".jpg");

		bStretch = false;
		image.load("face.jpg");
		resize(image.getPixels(), longestSide, ofVec2f(16, 9), bStretch, interp);
		image.setFromPixels(image.getPixels()); // ???
		image.save("face_169_crop_" + ofToString(i) + ".jpg");

		image.load("face.jpg");
		resize(image.getPixels(), longestSide, ofVec2f(9, 16), bStretch, interp);
		image.setFromPixels(image.getPixels()); // ???
		image.save("face_916_crop_" + ofToString(i) + ".jpg");
	}
}
@janimatic
Copy link
Contributor

Hello,
Thank you for your great work!
I am still experiencing the same crash with ofPixels::resizeTo (OF_INTERPOLATE_NEAREST_NEIGHBOR & OF_INTERPOLATE_BICUBIC, OF_INTERPOLATE_BILINEAR is not implemented).
In the meantime, i wrote a very simple method to generate proxies (since most of the time, resizing pixels is only used for performances, otherwise i'd use fbos for full image transformations).

template <typename PixelType>
ofPixels_<PixelType> fastResize(ofPixels_<PixelType> input, int proxyScale = 2)
{
	if (proxyScale < 2 || !(input.isAllocated())) return input;
	int dstWidth = input.getWidth() / proxyScale;
	int dstHeight = input.getHeight() / proxyScale;
	// create image for output  
	ofPixels_<PixelType> output;
	output.allocate(dstWidth, dstHeight, input.getPixelFormat());
	PixelType* dstPixels = output.getData();
	PixelType* srcPixels = input.getData();
	size_t srcWidth = input.getWidth();
	size_t srcHeight = input.getHeight();
	size_t dstIndex = 0;
	for (size_t dsty = 0; dsty < dstHeight; dsty++) {
		for (size_t dstx = 0; dstx < dstWidth; dstx++) {
			size_t pixelIndex = dstIndex * proxyScale;
			dstPixels[dstIndex] = srcPixels[pixelIndex];
			dstIndex++;
		}
	}
	return output;
}

It would be nice to integrate this in the ofPixel class...
cheers !
Julien

@johanjohan
Copy link
Author

Hey Julien, this was some time ago. I remember that Arturo had fixed some internal indexing vars to size_t (64 bit) at one point, the source of the issue. I cannot recall which OF version though. Probably of 0.10.1 with win10 vs2017. Which version do you work with? Does it appear at all sizes of images or above a 32bit threshold? Cheers M

@janimatic
Copy link
Contributor

janimatic commented May 4, 2024

Hello Michael,
I am using the latest master (cmake based project).
The crash happened only with ofPixels_<float>, ofPixels_<unsigned char> worked fine if i remember well.
I suspect there might eventually be something wrong in pointer's arithmetics and looping the colors using getBytesPerPixel() as limit, but i'd need to look at it deeper when i have time.
Thanks for your feedback, i'd keep you informed with more testings asap...

@dimitre
Copy link
Member

dimitre commented May 4, 2024

Hello @janimatic can you please write the minimal code example that can reproduce the issue and post here?
Thank you

@janimatic
Copy link
Contributor

Hello @dimitre @johanjohan
Here is a minimal code example...
thanks!

#include "ofMain.h"
template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	ofLogWarning() << proxy.getColor(0);
}

int main( ){
	int thumbnailProxy = 16;
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BILINEAR);
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BILINEAR);
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);
}

@janimatic
Copy link
Contributor

janimatic commented May 5, 2024

Hey @dimitre @johanjohan ,
I think
replacing line

size_t bytesPerPixel = getBytesPerPixel();

size_t bytesPerPixel = getBytesPerPixel();
by
size_t bytesPerPixel = channelsFromPixelFormat(pixelFormat);
would fix OF_INTERPOLATE_NEAREST_NEIGHBOR (renaming the variable accordingly of course)
sizeof(int) was ok but sizeof(float) 4 would make it fail. I don't really understand why the pointer would use the size of pixel type to walk through color channels...
I didn't check for OF_INTERPOLATE_BICUBIC though !
thanks

@janimatic
Copy link
Contributor

PS :
This seem to work (but should be tested)

//----------------------------------------------------------------------
template<typename PixelType>
bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMethod interpMethod) const {
	if (&dst == this) {
		return true;
	}

	if (!(isAllocated()) || !(dst.isAllocated()) || getBytesPerPixel() != dst.getBytesPerPixel()) return false;

	size_t srcWidth = getWidth();
	size_t srcHeight = getHeight();
	size_t dstWidth = dst.getWidth();
	size_t dstHeight = dst.getHeight();
	size_t bytesPerPixel = getBytesPerPixel();
	size_t channels = channelsFromPixelFormat(pixelFormat);

	PixelType* dstPixels = dst.getData();

	switch (interpMethod) {

		//----------------------------------------
	case OF_INTERPOLATE_NEAREST_NEIGHBOR: {
		size_t dstIndex = 0;
		float srcxFactor = (float)srcWidth / dstWidth;
		float srcyFactor = (float)srcHeight / dstHeight;
		float srcy = 0.5;
		for (size_t dsty = 0; dsty < dstHeight; dsty++) {
			float srcx = 0.5;
			size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
			for (size_t dstx = 0; dstx < dstWidth; dstx++) {
				size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channels;
				for (size_t k = 0; k < channels; k++) {
					dstPixels[dstIndex] = pixels[pixelIndex];
					dstIndex++;
					pixelIndex++;
				}
				srcx += srcxFactor;
			}
			srcy += srcyFactor;
		}
	}break;

		//----------------------------------------
	case OF_INTERPOLATE_BILINEAR:
		// not implemented yet
		ofLogError("ofPixels") << "resizeTo(): bilinear resize not implemented, not resizing";
		break;

		//----------------------------------------
	case OF_INTERPOLATE_BICUBIC:
		float px1, py1;
		float px2, py2;
		float px3, py3;

		float srcColor = 0;
		float interpCol;
		size_t patchRow;
		size_t patchIndex;
		float patch[16];

		size_t srcRowBytes = srcWidth * bytesPerPixel;
		size_t loIndex = (srcRowBytes)+1;
		//size_t hiIndex = (srcWidth * srcHeight * bytesPerPixel) - (srcRowBytes)-1;
		size_t hiIndex = (srcWidth * srcHeight * channels) - (srcRowBytes)-1;

		for (size_t dsty = 0; dsty < dstHeight; dsty++) {
			for (size_t dstx = 0; dstx < dstWidth; dstx++) {

				//size_t   dstIndex0 = (dsty * dstWidth + dstx) * bytesPerPixel;
				size_t   dstIndex0 = (dsty * dstWidth + dstx) * channels;
				float srcxf = srcWidth * (float)dstx / (float)dstWidth;
				float srcyf = srcHeight * (float)dsty / (float)dstHeight;
				size_t   srcx = static_cast<size_t>(std::min(srcWidth - 1, static_cast<size_t>(srcxf)));
				size_t   srcy = static_cast<size_t>(std::min(srcHeight - 1, static_cast<size_t>(srcyf)));
				//size_t   srcIndex0 = (srcy * srcWidth + srcx) * bytesPerPixel;
				size_t   srcIndex0 = (srcy * srcWidth + srcx) * channels;

				px1 = srcxf - srcx;
				py1 = srcyf - srcy;
				px2 = px1 * px1;
				px3 = px2 * px1;
				py2 = py1 * py1;
				py3 = py2 * py1;

				//for (size_t k = 0; k < bytesPerPixel; k++) {
				for (size_t k = 0; k < channels; k++) {
					size_t   dstIndex = dstIndex0 + k;
					size_t   srcIndex = srcIndex0 + k;

					for (size_t dy = 0; dy < 4; dy++) {
						patchRow = srcIndex + ((dy - 1) * srcRowBytes);
						for (size_t dx = 0; dx < 4; dx++) {
							//patchIndex = patchRow + (dx - 1) * bytesPerPixel;
							patchIndex = patchRow + (dx - 1) * channels;
							if ((patchIndex >= loIndex) && (patchIndex < hiIndex)) {
								srcColor = pixels[patchIndex];
							}
							patch[dx * 4 + dy] = srcColor;
						}
					}

					interpCol = (PixelType)bicubicInterpolate(patch, px1, py1, px2, py2, px3, py3);
					dstPixels[dstIndex] = interpCol;
				}

			}
		}
		break;
	}

	return true;
}

But with this test

#include "ofMain.h"
//========================================================================

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	ofLogWarning() << pixels.getColor(0);
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	ofLogWarning() << proxy.getColor(0);
}

int main() {
	int thumbnailProxy = 16;
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 255, 255));
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 255, 255));
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(1, 1, 1, 1));
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(1, 1, 1, 1));
}

i am facing strange logged values from a simple pixels.setColor(ofColor(1, 1, 1, 1)); on float pixels :

[warning] 255, 255, 255, 255
[warning] 255, 255, 255, 255
[warning] 255, 255, 255, 255
[warning] 0, 255, 255, 255
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0, 0, 0, 0

@dimitre
Copy link
Member

dimitre commented May 5, 2024

I'm proposing a simplification of channel / bytes handling here:

@dimitre
Copy link
Member

dimitre commented May 6, 2024

Please test the PR to see if it works for you @janimatic
there are lots of changes but the main one is this:

	pixelsSize = newSize / sizeof(PixelType);

and

	pixelsSize = bytesFromPixelFormat(w,h,_pixelFormat) / sizeof(PixelType);

we should not divide the pixelsSize by the sizeof(PixelType), it works with char because it is / 1 (memory size of char) but it will cause problems with float etc,
cc: @ofTheo @NickHardeman

@janimatic
Copy link
Contributor

Hello @dimitre ,
thank you very much for this.
For the moment, testing with float pixels,

  • if i use this minimal test, i get funky logged values
#include "ofMain.h"
//========================================================================

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	ofLogWarning() << pixels.getColor(0);
	ofLogWarning() << pixels.getColor(0).r << "," << pixels.getColor(0).g << "," << pixels.getColor(0).b << "," << pixels.getColor(0).a;
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	ofLogWarning() << proxy.getColor(0);
	ofLogWarning() << proxy.getColor(0).r << "," << proxy.getColor(0).g << "," << proxy.getColor(0).b << "," << proxy.getColor(0).a;
}

int main() {
	int thumbnailProxy = 16;
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 255, 255));
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 255, 255));
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(1, 1, 1, 1));
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(1, 1, 1, 1));
}
/*
outputs
[warning] 255, 255, 255, 255
[warning] ÿ,ÿ,ÿ,ÿ
[warning] 80, 1, 7, 76
[warning] P,�,�,L
[warning] 255, 255, 255, 255
[warning] ÿ,ÿ,ÿ,ÿ
[warning] 0, 255, 255, 255
[warning] [warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157,0.00392157,0.00392157,0.00392157
[warning] 3.53908e+07, 6.0396e-43, 3.55085e+07, 6.0396e-43
[warning] 3.53908e+07,6.0396e-43,3.55085e+07,6.0396e-43
[warning] 0.00392157, 0.00392157, 0.00392157, 0.00392157
[warning] 0.00392157,0.00392157,0.00392157,0.00392157
[warning] 0, 0, 0, 0
[warning] 0,0,0,0
*/

Is my testing method incorrect?
I guess i shouldn't complicate the testing with Qt but ...just for info

  • If i try tu use Qt6 to convert to QImage::Format_RGBA32FPx4 it doesn't output a QImage,
QTableWidgetItem* item = new QTableWidgetItem(text);
                    QImage image((const uchar *)pixels.getData(), pixels.getWidth(), pixels.getHeight(), QImage::Format_RGBA32FPx4);
                    item->setData(Qt::DecorationRole, QPixmap::fromImage(image));
                    item->setSizeHint(QSize(pixels.getWidth(), pixels.getHeight()));

  • If I use Qt5 compatible code and a silly pixel by pixel conversion (float to int)
    Qt print QImage::setPixelColor: color is invalid
                    QImage image(pixels.getWidth(), pixels.getHeight(), QImage::Format_ARGB32);
                    for (int y = 0; y < image.height(); y++) {
                        for (int x = 0; x < image.width(); x++) {
                            ofColor col = pixels.getColor(x, y);
                            image.setPixelColor(x, y, QColor(col.r * 255, col.g * 255, col.b * 255, col.a * 255));
                        }
                    }
                    item->setData(Qt::DecorationRole, QPixmap::fromImage(image));

I can create a tiny exr app quickly if that helps for a more concrete and visual testing.
Tell me how i can help.

PS : I once wrote a template class to avoid dealing with pixels pointers arithmetic wich drive me nut when writing openfx cpu image effect plugins, without any allocation unless (requested). It was used with float image (in fusion) and arbitrary data type.
Not sure if that usefull, it's not the best, but i post it here just in case ?

Thank you very much for your great work !

/// <summary>
/// An Image Class making color access easier
/// Using internal image data pointer, no copy occurs.
/// It just makes indexing easier
/// But we cannot use pointer arithmetic on it, since the data is still unidimensional (such as float* RGBARGBARGBA....)
/// </summary>
/// <typeparam name="CHANNELTYPE">any number type, openfx in resolve typically uses float images: auto srcColors = ColorImage<float>(src); </typeparam>

// USE_VEC seams to be broken now ? It would be nice to get rid of raw pointers though....
//#define USE_VEC
// vector is handy for debugging, but slightly slower :
// using CHANNELTYPE* _array, raw pointer makes no copy...
template <typename CHANNELTYPE>
class ColorImage {
private:
#ifdef USE_VEC
    std::vector<CHANNELTYPE> _array;
#else
    CHANNELTYPE* _array;
#endif // USE_VEC
    int _w, _h, _nComponents;
    u64 _index = 0;     // keep track of internal index : this is not realiable we user uses 
    int _channel = 0; // to interpret single channel, set this to desired channel index and just use pixels(x,y) method as usual...
    int _stride;
    bool internalArray = false;
public:
    ColorImage() {}
    // openfx image constructor
    ColorImage(OFX::Image* img) :
        _w(img->getBounds().x2 - img->getBounds().x1),
        _h(img->getBounds().y2 - img->getBounds().y1),
        _stride(img->getRowBytes()),
        _nComponents(img->getPixelComponentCount())
    {
        // TODO : make a true monochromatic copy in the constructor
        // https://www.cs.odu.edu/~zeil/cs330/latest/Public/big3/index.html
#ifdef USE_VEC
        auto ptr = static_cast<CHANNELTYPE*>(img->getPixelData());
        u64 arraySize = _w * _h * _nComponents;
        // fill the vector with OFX::Image data (float* pointer)
        _array.insert(_array.end(), ptr, ptr + arraySize);
#else
        _array = static_cast<CHANNELTYPE*>(img->getPixelData());
#endif // USE_VEC
    }
    // any imaga data pointer constructor
    ColorImage(CHANNELTYPE* ptr, int width, int height, int nComponents) {
        _array = ptr;
        _w = width;
        _h = height;
        _nComponents = nComponents;
        _calculateStride();
    }
    // copy constructor
    //ColorImage(ColorImage& other) {
    ColorImage(const ColorImage& other) {
        ColorImage tmp(other._array, other._w, other._h, other._nComponents);
    }
    ColorImage& operator=(const ColorImage& other)
    {
        this->_array = other._array;
        this->_w = other._w;
        this->_h = other._h;
        this->_nComponents = other._nComponents;
        return *this;
    }
    // initial allocation : provide the dimension, and the array of any type will be filled automatically...
    ColorImage(int width, int height, int nComponents) {
        _w = width;
        _h = height;
        _nComponents = nComponents;
        internalArray = true;
        u64 arraySize = _w * _h * _nComponents;
#ifdef USE_VEC
        // fill the vector with custom data (any* pointer)
        _array.resize(arraySize, CHANNELTYPE());
#else
        _array = new CHANNELTYPE[arraySize]();
        // Notice the parenthesis in new CHANNELTYPE[arraySize]() : 
        // Since CHANNELTYPE's constructor is called, there is no need to set initial data manually in a loop...
#endif // USE_VEC
        _calculateStride();
    }
    void copyPixelFrom(u64 idx, ColorImage<CHANNELTYPE>& from) {
        _array[_nComponents * idx + 0] = from._array[_nComponents * idx + 0];
        _array[_nComponents * idx + 1] = from._array[_nComponents * idx + 1];
        _array[_nComponents * idx + 2] = from._array[_nComponents * idx + 2];
        _array[_nComponents * idx + 3] = from._array[_nComponents * idx + 3];
    }
    void copyPixelFrom(int x, int y, ColorImage<CHANNELTYPE>& from) {
        _array[_nComponents * pixelIndex(_w, _h, x, y) + 0] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 0];
        _array[_nComponents * pixelIndex(_w, _h, x, y) + 1] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 1];
        _array[_nComponents * pixelIndex(_w, _h, x, y) + 2] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 2];
        _array[_nComponents * pixelIndex(_w, _h, x, y) + 3] = from._array[_nComponents * pixelIndex(_w, _h, x, y) + 3];
    }
    void _calculateStride() {
        // we don't need stride in bytes : the float* pointer addresses elements, not bytes...
        // see https://medium.com/@oleg.shipitko/what-does-stride-mean-in-image-processing-bba158a72bcd
        // see https://cplusplus.com/forum/windows/162811/
        // we keep this here just for reference.
        // This result seems to be conform with OFX::Image getRowBytes() result anyway...
        _stride = _w * sizeof(CHANNELTYPE) * _nComponents;
        // No padding ? Not sure of this...
        //_stride = ((_stride + 63) / 64) * 64;
    }
    int getNumberOfChannels() {
        return _nComponents;
    }
    int bytesPerRow() {
        return _stride;
    }
    ~ColorImage() {
        if (internalArray) {
#ifdef USE_VEC
            _array.clear();
#else
            // rewind before delete ?
            // https://stackoverflow.com/questions/41184086/why-does-my-program-crash-when-i-increment-a-pointer-and-then-delete-it
            _array = 0;
            delete[] _array;
#endif // USE_VEC
        }
        //if (internalArray) free(_array);
    }
    bool isAllocated() {
        return _array != nullptr;
        //return _array != nullptr && _channel > 0;
    }
    void release() {
#ifdef USE_VEC
        _array.clear();
#else
        _array = 0;
        delete[] _array;
#endif // USE_VEC
    }
    // reset to default value
    void reset(){
        u64 arraySize = _w * _h * _nComponents;
        for (auto i=0; i<arraySize; i++)
            _array[i] = CHANNELTYPE();
    }
    int getWidth() { return _w; }
    int getHeight() { return _h; }

    // toonz compatibility
    int getLx() { return _w; }
    int getLy() { return _h; }
    int getWrap() { return _w; }
    // to use for channel access 
    void setChannel(int c) {
        // for now, we just pass an int index to define the channel
        // TODO:
        // a bitset https://en.cppreference.com/w/cpp/utility/bitset or 4 booleans or one string ?
        // make sure it's very efficient... : it's just an offset: r:0 g:1 b:2 a:4
        assert(c <= _nComponents && c >= 0);
        if (c > _nComponents) c = _nComponents;
        if (c < 0) c = 0;
        _channel = c;
    }
    int channel() { return _channel; }
    // returns a pointer address with offset
    CHANNELTYPE* pixels(u64 idx) {
        assert(idx < _w * _h && idx >= 0);
        if (idx > _w * _h) idx = _w * _h;
        if (idx < 0 ) idx = 0;
        // https://stackoverflow.com/questions/6485496/how-to-get-stdvector-pointer-to-the-raw-data
        return &_array[_nComponents * idx + _channel];
        //return _array + (_nComponents * idx + 0);
    }
    // returns a pointer address with offset
    // global pointer offset is equivalent to red offset (first index of the pixel)
#ifdef USE_VEC
    //	Erro C2440	'return' : impossible de convertir de 'const _Ty *' en 'CHANNELTYPE *'
    CHANNELTYPE* pixels(int x, int y)  {
#else
    CHANNELTYPE* pixels(int x, int y) const {
#endif // USE_VEC
        // clamp x & y, if necessary
        if (x >= _w) x = _w - 1;
        if (x < 0) x = 0;
        if (y >= _h) y = _h - 1;
        if (y < 0) y = 0;
        // https://stackoverflow.com/questions/6485496/how-to-get-stdvector-pointer-to-the-raw-data
        return &_array[_nComponents * pixelIndex(_w, _h, x, y) + _channel]; // _channel : monochromatic access...
    }
    bool areEqual(u64 idxA, u64 idxB, CHANNELTYPE threshold) const {
        auto a = pixel(idxA);
        auto b = pixel(idxB);
        return std::max({ abs((CHANNELTYPE)a.r - b.r), abs((CHANNELTYPE)a.g - b.g),
                         abs((CHANNELTYPE)a.b - b.b), abs((CHANNELTYPE)a.a - b.a) }) < threshold;
    }

    CHANNELTYPE* getPreviousPixel(CHANNELTYPE* ptr) {
        return ptr - _nComponents;
    }
    CHANNELTYPE* getNextPixel(CHANNELTYPE* ptr) {
        return ptr + _nComponents;
    }
    CHANNELTYPE* getPreviousRow(CHANNELTYPE* ptr) {
        // we don't need stride in bytes : the float* pointer addresses elements, not bytes...
        // return reinterpret_cast<CHANNELTYPE*>(reinterpret_cast<std::uint8_t*>(ptr) - _stride);
        return ptr - (_w * _nComponents);
    }
    CHANNELTYPE* getNextRow(CHANNELTYPE* ptr) {
        // return reinterpret_cast<CHANNELTYPE*>(reinterpret_cast<std::uint8_t*>(ptr) + _stride);
        return ptr + (_w * _nComponents);
    }

    // unidimensional pixel indexing (i)
    // Safe access to pixel's components (rgba) :
    // It returns 0 if the components doesn't exit
    CHANNELTYPE r(u64 idx) {
        //assert(_nComponents > 0);
        if (_nComponents > 0)
            return _array[_nComponents * idx + 0];
        return 0.0;
    }
    CHANNELTYPE g(u64 idx) {
        //assert(_nComponents > 1);
        if (_nComponents > 1)
            return _array[_nComponents * idx + 1];
        return 0.0;
    }
    CHANNELTYPE b(u64 idx) {
        //assert(_nComponents > 2);
        if (_nComponents > 2)
            return _array[_nComponents * idx + 2];
        return 0.0;
    }
    CHANNELTYPE a(u64 idx) {
        //assert(_nComponents > 3);
        if (_nComponents > 3)
            return _array[_nComponents * idx + 3];
        return 0.0;
    }
    RgbaPixel<CHANNELTYPE> pixel(u64 idx) {
        return RgbaPixel<CHANNELTYPE> {r(idx), g(idx), b(idx), a(idx)};
    }
    // bidimensional pixel indexing (x,y)
    // Safe access to pixel's components (rgba) :
    // It returns 0 if the components doesn't exit
    CHANNELTYPE r(int x, int y) {
        return r(pixelIndex(_w, _h, x, y));
    }
    CHANNELTYPE g(int x, int y) {
        return g(pixelIndex(_w, _h, x, y));
    }
    CHANNELTYPE b(int x, int y) {
        return b(pixelIndex(_w, _h, x, y));
    }
    CHANNELTYPE a(int x, int y) {
        return a(pixelIndex(_w, _h, x, y));
    }
    RgbaPixel<CHANNELTYPE> pixel(int x, int y) {
        return pixel(pixelIndex(_w, _h, x, y));
    }
    // unidimensional pixel indexing (i)
    // Safe pixel's components setter (rgba) :
    // Skipping if the components doesn't exit
    void setR(CHANNELTYPE v, u64 idx) {
        //assert(_nComponents > 0);
        if (_nComponents > 0)
            _array[_nComponents * idx + 0] = v;
    }
    void setG(CHANNELTYPE v, u64 idx) {
        //assert(_nComponents > 1);
        if (_nComponents > 1)
            _array[_nComponents * idx + 1] = v;
    }
    void setB(CHANNELTYPE v, u64 idx) {
        //assert(_nComponents > 2);
        if (_nComponents > 2)
            _array[_nComponents * idx + 2] = v;
    }
    void setA(CHANNELTYPE v, u64 idx) {
        //assert(_nComponents > 3);
        if (_nComponents > 3)
            _array[_nComponents * idx + 3] = v;
    }
    void setPixel(CHANNELTYPE r, CHANNELTYPE g, CHANNELTYPE b, CHANNELTYPE a, u64 idx) {
        setR(r, idx);
        setG(g, idx);
        setB(b, idx);
        setA(a, idx);
    }
    void setPixel(RgbaPixel<CHANNELTYPE> pixel, u64 idx) {
        setPixel(pixel.r, pixel.g, pixel.b, pixel.a, idx);
    }
    // bidimensional pixel indexing (x,y)
    // Safe pixel's components setter (rgba) :
    // Skiping if the components doesn't exit
    void setR(CHANNELTYPE v, int x, int y) {
        setR(v, pixelIndex(_w, _h, x, y));
    }
    void setG(CHANNELTYPE v, int x, int y) {
        setG(v, pixelIndex(_w, _h, x, y));
    }
    void setB(CHANNELTYPE v, int x, int y) {
        setB(v, pixelIndex(_w, _h, x, y));
    }
    void setA(CHANNELTYPE v, int x, int y) {
        setA(v, pixelIndex(_w, _h, x, y));
    }
    void setPixel(CHANNELTYPE r, CHANNELTYPE g, CHANNELTYPE b, CHANNELTYPE a, int x, int y) {
        setPixel(r, g, b, a, pixelIndex(_w, _h, x, y));
    }
    void setPixel(RgbaPixel<CHANNELTYPE> pixel, int x, int y) {
        setPixel(pixel, pixelIndex(_w, _h, x, y));
    }
    // unidimensional pixel indexing (i)
    CHANNELTYPE* operator[](u64 index) const
    {
        return &this->_array[_nComponents * index];
    }
    // raw data getter
    CHANNELTYPE* data()
    {
#ifdef USE_VEC
        return _array.data();
#else
        return _array;
#endif // USE_VEC
    }
    // raw data setter
    void setData(CHANNELTYPE* data)
    {
        _array = data;
    }
};

@dimitre
Copy link
Member

dimitre commented May 6, 2024

I didnt' read everything yet, but some suggestions in your code:
1 - if you convert to int before log it won't display chars.
ofLogWarning() << int(pixels.getColor(0).r) << "," << int(pixels.getColor(0).g) << "," << int(pixels.getColor(0).b) << "," << int(pixels.getColor(0).a);

2 - ofColor(1, 1, 1, 1) is almost black. maybe you wanted ofFloatColor(1, 1, 1, 1)

@dimitre
Copy link
Member

dimitre commented May 6, 2024

And I've changed the NEAREST_NEIGHBOR code, but didn't test it. if you find it is wrong you can revert to OF core one.

@dimitre
Copy link
Member

dimitre commented May 6, 2024

it seems to be working here :

#include "ofMain.h"
//========================================================================

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	{
		ofColor color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogWarning() << color;
	}
//	ofLogWarning() << pixels.getColor(0);
//	ofLogWarning() << int(pixels.getColor(0).r) << "," << int(pixels.getColor(0).g) << "," << int(pixels.getColor(0).b) << "," << int(pixels.getColor(0).a);
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	{
		ofColor color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogWarning() << color;
	}
//	ofLogWarning() << color;
//	ofLogWarning() << int(proxy.getColor(0).r) << "," << int(proxy.getColor(0).g) << "," << int(proxy.getColor(0).b) << "," << int(proxy.getColor(0).a);
}

int main() {
	int thumbnailProxy = 16;
	cout << "--- one" << endl;
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
	cout << "--- two" << endl;
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
	cout << "--- three" << endl;
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
	cout << "--- four" << endl;
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));
}

@janimatic
Copy link
Contributor

janimatic commented May 6, 2024

@dimitre
great!
i just modified it with templated color and it seem to work with floats...perfect...

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogWarning() << color;
	}
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogWarning() << color;
	}
}
/*
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 255, 255, 0, 255
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
[warning] 1, 1, 0, 1
*/

@janimatic
Copy link
Contributor

@dimitre
for the moment, the resize seem broken with 8 bit pixels too (just tested in the context of Qt thumbnail generation that works in master, I didn't write a simple test case with picture. Hopefully i'll do when i get time...)
thanks

@dimitre
Copy link
Member

dimitre commented May 6, 2024

Thanks, let me know when you have the tests. you can try a minimal change to the core by finding "pixelsSize = " and removing sizeof()

@janimatic
Copy link
Contributor

hello @dimitre
Your small fix suggestion on master seems perfect.
For the refactor pr testing, I wrote some test code for float images io...
It could not be very minimal though, since minimal float image io is not that small...
Here is a test using tinyexr and stbimage libs + ofImage (which internally resizes using freeimage, and can read/write 32 bit tifs)
I've attached the full source + image data
007_imageio.zip
Thanks !

#include "ofMain.h"
//========================================================================
#include "imageIO.h"

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
}

template <typename PixelType>
void resizeIOTest_ofImage(int thumbnailProxy, string prefix, string extension) {
	ofLogNotice() << "ofImage_<float> test (using freeimage resize)";
	ofImage_<PixelType> floatimage;
	auto filename = prefix + "." + extension;
	if (!floatimage.load(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while loading ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while loading ofImage (using freeimage) " << filename;
	filename = prefix + "_copy." + extension;
	if (!floatimage.save(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
	floatimage.resize(floatimage.getWidth() / thumbnailProxy, floatimage.getHeight() / thumbnailProxy);
	filename = prefix + "_proxy." + extension;
	if (!floatimage.save(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
}

template <typename PixelType>
void resizeIOTest_ofPixels(int thumbnailProxy, string prefix, string extension, ofInterpolationMethod interp) {
	ofLogNotice() << "ofPixels test using tinyexr & stb to load and save float images and proxies";
	int width, height, channels;
	auto filename = prefix + "." + extension;
	PixelType* data = ImageIO::loadImage(ofToDataPath(filename, true), width, height, channels);
	ofPixels_<PixelType> pixels;
	pixels.setFromPixels(data, width, height, OF_IMAGE_COLOR_ALPHA);
	filename = prefix + "_copy." + extension;
	ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
	pixels.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	width = pixels.getWidth() / thumbnailProxy;
	height = pixels.getHeight() / thumbnailProxy;
	filename = prefix + "_proxy." + extension;
	ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
}

int main() {
	int thumbnailProxy = 16;
	ofLogNotice();
	ofLogNotice() << "--- one";
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
	ofLogNotice();
	ofLogNotice() << "--- two";
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
	ofLogNotice();
	ofLogNotice() << "--- three";
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
	ofLogNotice();
	ofLogNotice() << "--- four";
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));

	ofLogNotice();
	ofLogNotice() << "convert pixels int to float with ofPixels =";
	ofPixels_<unsigned char> pixels;
	pixels.allocate(1920, 1080, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(ofColor(255, 255, 0, 255));
	{
		ofColor_<unsigned char> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
	ofPixels_<float> floatpixels;
	floatpixels = pixels;
	auto color = floatpixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
	ofLogNotice() << color;

	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear tif resizeIOTest_ofImage";
	resizeIOTest_ofImage<float>(thumbnailProxy, "Digital_LAD_HD720", "tif");
	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_NEAREST_NEIGHBOR";
	resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_BICUBIC";
	resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);

}

@dimitre
Copy link
Member

dimitre commented May 7, 2024

@janimatic can you please test this other PR?
#7936

@janimatic
Copy link
Contributor

@dimitre hello!

  • the Qt thumbs are working with int anf float pixels (Qt6 QImage::Format_RGBA32FPx4 thumb looks ultra pixalated.)
  • ofImage_ / freeimage resize removes the alpha channel
  • this test code generates a pixelated exr (compared to the tif generated by ofImage_ / freeimage resize)
    I am not sure if that's my code of the ofPixel resize that is broken (looks like a wrong channel offset).
    But since ofpixels to Qt6 thumb looks pixelated too i guess it's of resize that's still broken.
    (see ofimage_resize_tif.png vs ofpixel_resize_exr.png)
    (Note : my previous test code was plain wrong : it divide by thumbnailProxy twice)
ofimage_resize_tif ofpixel_resize_exr
#include "ofMain.h"
//========================================================================
#include "imageIO.h"

template <typename PixelType>
void resizeTest(int width, int height, int thumbnailProxy, ofInterpolationMethod interp, ofColor_<PixelType> col) {
	ofPixels_<PixelType> pixels;
	pixels.allocate(width, height, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(col);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
	auto proxy = pixels;
	proxy.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	{
		ofColor_<PixelType> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
}

template <typename PixelType>
void resizeIOTest_ofImage(int thumbnailProxy, string prefix, string extension) {
	ofLogNotice() << "ofImage_<float> test (using freeimage resize)";
	ofImage_<PixelType> floatimage;
	auto filename = prefix + "." + extension;
	if (!floatimage.load(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while loading ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while loading ofImage (using freeimage) " << filename;
	filename = prefix + "_copy." + extension;
	if (!floatimage.save(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
	floatimage.resize(floatimage.getWidth() / thumbnailProxy, floatimage.getHeight() / thumbnailProxy);
	filename = prefix + "_proxy." + extension;
	if (!floatimage.save(ofToDataPath(filename, true)))
		ofLogWarning() << "failed while saving ofImage (using freeimage) " << filename;
	else
		ofLogNotice() << "success while saving ofImage (using freeimage) " << filename;
}

template <typename PixelType>
void resizeIOTest_ofPixels(int thumbnailProxy, string prefix, string extension, ofInterpolationMethod interp) {
	ofLogNotice() << "ofPixels test using tinyexr & stb to load and save float images and proxies";
	int width, height, channels;
	auto filename = prefix + "." + extension;
	PixelType* data = ImageIO::loadImage(ofToDataPath(filename, true), width, height, channels);
	ofLogNotice() << "ImageIO::loadImage width: " << width << " height: "<< height << " channels: " << channels;
	ofPixels_<PixelType> pixels;
	pixels.setFromPixels(data, width, height, OF_IMAGE_COLOR_ALPHA);
	filename = prefix + "_copy." + extension;
	ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
	pixels.resize(pixels.getWidth() / thumbnailProxy, pixels.getHeight() / thumbnailProxy, interp);
	width = pixels.getWidth();
	height = pixels.getHeight();
	filename = prefix + "_proxy." + extension;
	ofLogNotice() << "saving resized proxy with width: " << width << " height: " << height << " channels: " << channels << " extension: " << extension;
	ImageIO::saveImage(ofToDataPath(filename, true), width, height, channels, pixels.getData());
}

int main() {
	int thumbnailProxy = 10;
	ofLogNotice();
	ofLogNotice() << "--- one";
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofColor(255, 255, 0, 255));
	ofLogNotice();
	ofLogNotice() << "--- two";
	resizeTest<unsigned char>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofColor(255, 255, 0, 255));
	ofLogNotice();
	ofLogNotice() << "--- three";
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR, ofFloatColor(1, 1, 0, 1));
	ofLogNotice();
	ofLogNotice() << "--- four";
	resizeTest<float>(1920, 1080, thumbnailProxy, ofInterpolationMethod::OF_INTERPOLATE_BICUBIC, ofFloatColor(1, 1, 0, 1));

	ofLogNotice();
	ofLogNotice() << "convert pixels int to float with ofPixels =";
	ofPixels_<unsigned char> pixels;
	pixels.allocate(1920, 1080, OF_IMAGE_COLOR_ALPHA);
	pixels.setColor(ofColor(255, 255, 0, 255));
	{
		ofColor_<unsigned char> color = pixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
		ofLogNotice() << color;
	}
	ofPixels_<float> floatpixels;
	floatpixels = pixels;
	auto color = floatpixels.getColor(pixels.getWidth() * .5, pixels.getHeight() * .5);
	ofLogNotice() << color;

	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear tif resizeIOTest_ofImage";
	resizeIOTest_ofImage<float>(thumbnailProxy, "Digital_LAD_HD720", "tif");
	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_NEAREST_NEIGHBOR";
	resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_NEAREST_NEIGHBOR);
	ofLogNotice();
	ofLogNotice() << "--- 32 bits linear exr resizeIOTest_ofPixels OF_INTERPOLATE_BICUBIC";
	//resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720", "exr", ofInterpolationMethod::OF_INTERPOLATE_BICUBIC);

}

  • maybe we should use channels to walk though channels channelsFromPixelFormat(pixelFormat);
		case OF_INTERPOLATE_NEAREST_NEIGHBOR:{
			size_t dstIndex = 0;
			float srcxFactor = (float)srcWidth/dstWidth;
			float srcyFactor = (float)srcHeight/dstHeight;
			float srcy = 0.5;
			for (size_t dsty=0; dsty<dstHeight; dsty++){
				float srcx = 0.5;
				size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
				for (size_t dstx=0; dstx<dstWidth; dstx++){
					//size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
					//for (size_t k=0; k<bytesPerPixel; k++){
					size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channelsFromPixelFormat(pixelFormat);
					for (size_t k=0; k< channelsFromPixelFormat(pixelFormat); k++){
						dstPixels[dstIndex] = pixels[pixelIndex];
						dstIndex++;
						pixelIndex++;
					}
					srcx+=srcxFactor;
				}
				srcy+=srcyFactor;
			}
		}break;

That looks correct to me (see ofpixel_resize_exrPatch.png)

ofpixel_resize_exrPatch

I didn't check the OF_INTERPOLATE_BICUBIC...

Thank you !

@janimatic
Copy link
Contributor

  • This is the current exr output from bicubic resize :
ofpixel_resize_exrBicubic
  • This is the exr output from bicubic resize when i replace bytesPerPixel by channelsFromPixelFormat(pixelFormat) :
ofpixel_resize_exrBicubicPatch

That fixed the pixels indices, but bicubic must have something wrong with accumulated colors in float...

@janimatic
Copy link
Contributor

janimatic commented May 8, 2024

@dimitre
This the result of exr bicubic resize, if i remove the clamping code from ofPixels_::bicubicInterpolate:

	//return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
	return out;
ofpixel_resize_exrBicubicPatch2

@janimatic
Copy link
Contributor

janimatic commented May 8, 2024

To summarize, here is a diff of my changes made in #7936

diff --git a/libs/openFrameworks/graphics/ofPixels.cpp b/libs/openFrameworks/graphics/ofPixels.cpp
index 44a3f926e..d23f4eeaa 100644
--- a/libs/openFrameworks/graphics/ofPixels.cpp
+++ b/libs/openFrameworks/graphics/ofPixels.cpp
@@ -1328,7 +1328,8 @@ float ofPixels_<PixelType>::bicubicInterpolate (const float *patch, float x,floa
 	a20 * x2 + a21 * x2 * y + a22 * x2 * y2 + a23 * x2 * y3 +
 	a30 * x3 + a31 * x3 * y + a32 * x3 * y2 + a33 * x3 * y3;
 
-	return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
+	//return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));
+	return out;
 }
 
 //----------------------------------------------------------------------
@@ -1361,8 +1362,10 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
 				float srcx = 0.5;
 				size_t srcIndex = static_cast<size_t>(srcy) * srcWidth;
 				for (size_t dstx=0; dstx<dstWidth; dstx++){
-					size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
-					for (size_t k=0; k<bytesPerPixel; k++){
+					//size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * bytesPerPixel;
+					//for (size_t k=0; k<bytesPerPixel; k++){
+					size_t pixelIndex = static_cast<size_t>(srcIndex + srcx) * channelsFromPixelFormat(pixelFormat);
+					for (size_t k=0; k< channelsFromPixelFormat(pixelFormat); k++){
 						dstPixels[dstIndex] = pixels[pixelIndex];
 						dstIndex++;
 						pixelIndex++;
@@ -1391,19 +1394,19 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
 			size_t patchIndex;
 			float patch[16];
 
-			size_t srcRowBytes = srcWidth*bytesPerPixel;
+			size_t srcRowBytes = srcWidth*channelsFromPixelFormat(pixelFormat);
 			size_t loIndex = (srcRowBytes)+1;
-			size_t hiIndex = (srcWidth*srcHeight*bytesPerPixel)-(srcRowBytes)-1;
+			size_t hiIndex = (srcWidth*srcHeight*channelsFromPixelFormat(pixelFormat))-(srcRowBytes)-1;
 
 			for (size_t dsty=0; dsty<dstHeight; dsty++){
 				for (size_t dstx=0; dstx<dstWidth; dstx++){
 
-					size_t   dstIndex0 = (dsty*dstWidth + dstx) * bytesPerPixel;
+					size_t   dstIndex0 = (dsty*dstWidth + dstx) * channelsFromPixelFormat(pixelFormat);
 					float srcxf = srcWidth  * (float)dstx/(float)dstWidth;
 					float srcyf = srcHeight * (float)dsty/(float)dstHeight;
 					size_t   srcx = static_cast<size_t>(std::min(srcWidth-1, static_cast<size_t>(srcxf)));
 					size_t   srcy = static_cast<size_t>(std::min(srcHeight-1, static_cast<size_t>(srcyf)));
-					size_t   srcIndex0 = (srcy*srcWidth + srcx) * bytesPerPixel;
+					size_t   srcIndex0 = (srcy*srcWidth + srcx) * channelsFromPixelFormat(pixelFormat);
 
 					px1 = srcxf - srcx;
 					py1 = srcyf - srcy;
@@ -1412,14 +1415,14 @@ bool ofPixels_<PixelType>::resizeTo(ofPixels_<PixelType>& dst, ofInterpolationMe
 					py2 = py1 * py1;
 					py3 = py2 * py1;
 
-					for (size_t k=0; k<bytesPerPixel; k++){
+					for (size_t k=0; k<channelsFromPixelFormat(pixelFormat); k++){
 						size_t   dstIndex = dstIndex0+k;
 						size_t   srcIndex = srcIndex0+k;
 
 						for (size_t dy=0; dy<4; dy++) {
 							patchRow = srcIndex + ((dy-1)*srcRowBytes);
 							for (size_t dx=0; dx<4; dx++) {
-								patchIndex = patchRow + (dx-1)*bytesPerPixel;
+								patchIndex = patchRow + (dx-1)*channelsFromPixelFormat(pixelFormat);
 								if ((patchIndex >= loIndex) && (patchIndex < hiIndex)) {
 									srcColor = pixels[patchIndex];
 								}

@janimatic
Copy link
Contributor

PS :
sorry for the noise, the latest change for bicubic is not good (it just looks like nearest neighbor with some offset)
Just removing the clamping didn't fix it...

@dimitre
Copy link
Member

dimitre commented May 8, 2024

did you paste your latest code here? I've run and I think there is a problem with the alpha channel on the exr copy
Screenshot 2024-05-08 at 08 22 18

@janimatic
Copy link
Contributor

in the the zip file ,
the main.cpp should be replaced with the code from #6226 (comment)
Other than that i've just adapted the openframeworks source like i said.
What are you using to display the exr ? (i use natron)

@dimitre
Copy link
Member

dimitre commented May 8, 2024

Screenshot 2024-05-08 at 08 28 19

opening the copy here in macOS finder and pixelmator

@janimatic
Copy link
Contributor

@dimitre
indeed the sample image had a luminance like alpha channel...
here is the source exr with white alpha sorry!
Uploading Digital_LAD_HD720.zip…

@janimatic
Copy link
Contributor

janimatic commented May 8, 2024

oops, you're right the copy doesn't copy the correct channel for alpha...
alpha=blue...

@dimitre
Copy link
Member

dimitre commented May 8, 2024

new image link appear as uploading here
if you agree this pr corrects the main issue (artifacts & crash with resize) I'll be merging it
#7936

@janimatic
Copy link
Contributor

janimatic commented May 8, 2024

well i am not sure it works at all in float if not using channels instead of bytes...
But we can always complete the fixes later if you prefer
By the way my exr code (ImageIO) adds a problem (blue channel on alpha output...) :

bool saveImageEXR_rgba(const char* outfilename, int width, int height, const float* rgba) {
    std::cout << "saveImageEXR_rgba file " << outfilename << std::endl;
    EXRHeader header;
    InitEXRHeader(&header);
    EXRImage image;
    InitEXRImage(&image);
    image.num_channels = 4;
    std::vector<float> images[4];
    images[0].resize(width * height);
    images[1].resize(width * height);
    images[2].resize(width * height);
    images[3].resize(width * height);

    // Split RGBARGBARGBA... into R, G, B, A layers
    for (int i = 0; i < width * height; i++) {
        images[0][i] = rgba[image.num_channels * i + 0];
        images[1][i] = rgba[image.num_channels * i + 1];
        images[2][i] = rgba[image.num_channels * i + 2];
        images[3][i] = rgba[image.num_channels * i + 3]; // 2]; NO! 
    }

@dimitre
Copy link
Member

dimitre commented May 8, 2024

Screenshot 2024-05-08 at 09 03 50

@janimatic
Copy link
Contributor

Digital_LAD_HD720.zip

@dimitre
Copy link
Member

dimitre commented May 8, 2024

Yes I was suggesting mergin only the pixelsSize issue for now, so I'll be able to test on master later.

@dimitre
Copy link
Member

dimitre commented May 8, 2024

now copy looks OK 🖼️

@janimatic
Copy link
Contributor

yes that's fine for me if you prefer to merge only the pixelsSize for now, even if we know that it creates a channel offset in resize.
For the additional changes suggested here, only the bicubic resize of float images remains to be fixed i think.
Thanks!

@janimatic
Copy link
Contributor

@dimitre
Hello, if you want to keep the unsigned char pixels code intact, you could keep the clamp for all template types for the return of
float ofPixels_::bicubicInterpolate like this :

	if (std::is_floating_point<PixelType>::value)
		return std::min(1.0f, std::max(out, 0.0f));
	return std::min(static_cast<size_t>(255), std::max(static_cast<size_t>(out), static_cast<size_t>(0)));

That ensure that there won't be any regression.
I suggest applying my patch ,
plus that change, in you pr, before merging...
And then we can try to see if bicubic can be enhanced in float images.
Thanks!

@dimitre
Copy link
Member

dimitre commented May 9, 2024

@janimatic yes, PR updated.
just to be sure, is it possible to use PixelType double?

@dimitre
Copy link
Member

dimitre commented May 9, 2024

@janimatic I suggest two separate PRs.
we can keep this one for just sizeof and clamping, merging and you can submit another with resize corrections

@dimitre
Copy link
Member

dimitre commented May 11, 2024

Hello @janimatic PR was just merged. Feel free to open another PR with resize fixes

@dimitre
Copy link
Member

dimitre commented May 21, 2024

@janimatic can you please make a PR with your patch, so we can test it ?

@janimatic
Copy link
Contributor

Hello @dimitre ,
I just did #7989
Eventually some variables names might still be confusing (color indices vs pixels indices) ?
Thanks

@johanjohan
Copy link
Author

hello guys, thank you for fixing this after such a long time.

@dimitre
Copy link
Member

dimitre commented May 22, 2024

Now merged. can you please test with the latest master to see if this issue can be closed?
Thank you all

@dimitre
Copy link
Member

dimitre commented May 22, 2024

@janimatic I've added your code for testing exr, tiff, png images with some changes to this repo
https://github.com/dimitre/ofTests
and invited you as a collaborator

it would be great if we could maybe support some different bit depth in ofImage.
I added two more files, exported from the .exr file from Pixelmator, a 16bit png and one HDR png (I didn't know it existed)

EDIT: feel free to push changes there

@danoli3
Copy link
Member

danoli3 commented May 23, 2024

This is wicked!!! Yeah let's get CI on this ofTests.
I has forked

OpenEXR issues looking patched on FreeImage SVN upstream, I'll port those into the Patched Repo.
It was just not compatible with the super old LucasArts code, which is updated on Github, just subdependancy issue with FreeImage. FreeImage had to have some subdepends disabled due to incompatibilities to C++14/17 and compile targets.

@janimatic
Copy link
Contributor

janimatic commented May 23, 2024

@dimitre
excellent thanks!
Some ideas of things we could add tests for :

  • BMP, TGA, JPEG, HDR stb image write
  • JPG, TGA, BMP, PSD (comp), GIF, HDR, PIC stb image read
  • exr with different compressions / tiles etc...
  • The bicubic resize didn't look as smooth as the freeimage one if i remember...
  • In the end it would be great integrate those high dynamic range image i/o to openframeworks base ? complementing FreeImage with TinyExr and StbImage...
  • short pixels. I never use it though...
  • I have been using custom/arbitrary structures pixels for some algorithms, but no i/o since it's not standard... (example: mlaa)
  • For info Stb has a (scary 10000 lines, but maybe usefull for reference) resize single header lib... https://github.com/nothings/stb
  • Do you consider using cmake ? That would probably make cross-platform testing easier.
    Cheers!
    PS there are many other formats used in the industry like dpx etc... but that's a lot of work that has been done already... https://github.com/AcademySoftwareFoundation/OpenImageIO

@dimitre
Copy link
Member

dimitre commented May 23, 2024

Great! it will be amazing if OF could handle more image formats and bit depth.
I would love to have HDR io in OF also, and +1 for adding other libs to handle HDR files (if licensing is OK)

Even simple tests like opening your tiff file is failing, so maybe there is more work needed on OF core

@dimitre
Copy link
Member

dimitre commented May 23, 2024

I think the place to have a look is putBmpIntoPixels function on OF Core.

@NickHardeman
Copy link
Contributor

@dimitre, HDR loading should be supported, not sure about saving though.
See https://github.com/openframeworks/openFrameworks/tree/master/examples/gl/materialPBRExample

@dimitre
Copy link
Member

dimitre commented May 23, 2024

yess. I suppose it can be done now allocating the right kind of ofPixels and using ofLoadImage right?
if we use ofImage_<float> instead it fails when allocating texture

EDIT: My bad, there was no GLFW window on tests, so no OpenGL context for textures

@NickHardeman
Copy link
Contributor

Yeah, check out here for loading into a texture.

I had to do some checks for OPENGL_ES since some versions or platforms didn't like GL_RGB32F and values over a certain limit would cause strange issues.

float* fPixData = fpix.getData();
unsigned int numFs = fw * fh * fpix.getNumChannels();
for( unsigned int i = 0; i < numFs; i++ ) {
	// clamp to the maximum value of float16, medium precision
	if(fPixData[i] > 65504.f) {
		fPixData[i] = 65504.f;
	}
}

@NickHardeman
Copy link
Contributor

Using ofLoadImage( texture, "img.hdr" ) to load a texture should be supported as well.

bool ofLoadImage(ofTexture & tex, const of::filesystem::path& path, bool bFlipInY, const ofImageLoadSettings &settings){
bool loaded = false;
std::string ext = ofToLower(path.extension().string());
bool hdr = (ext == ".hdr" || ext == ".exr");
if( hdr ) {
ofFloatPixels pixels;
loaded = ofLoadImage(pixels, path, settings);
if(loaded){
#if defined(TARGET_OPENGLES)
// GL_RGB32F, GL_RGBA32F and GL_RGB16F is not supported in Emscripten opengl es, so we need to set to GL_RGBA16F or GL_RGBA32F. But GL_RGBA32F is not supported via opengl es on most mobile devices as of right now.
if(pixels.getNumChannels() != 4 ) {
// set alpha to 1.
ofLogVerbose("ofLoadImage") << "changing number of loaded pixel channels from " << pixels.getNumChannels() << " to 4 for more broad support on OpenGL ES.";
pixels.setImageType( OF_IMAGE_COLOR_ALPHA );
}
#endif
if(bFlipInY) {
pixels.mirror(true, false);
}
tex.loadData(pixels);
}
} else {

@dimitre
Copy link
Member

dimitre commented May 23, 2024

@janimatic I've updated your tests and pushed code to ofTests repo.
it seems 16bit resize is still not working
Digital_LAD_HD720_16bit_copy

@janimatic
Copy link
Contributor

@dimitre
nice find, i'll take a look asap!
Talking about the current hdr support in ofImage, I believe (I might be wrong) :

  • FreeImage supports exr but not Multilayer exr, not compressed exr. (it should be tested). piz / zip are very usefull though, pretty fast in nuke and well compressed.
  • I believe it also supports 16 bit png
  • Tif is a different case because it can be 1 bit , 8 bit 16 bit or 32 bit per channel.
  • I am sure adding tinyexr + stb will cover more cases the freeimage misses, if you prefer small header only libs... But still not complete (missing dng, dpx etc..) openimageio is probably the best...
  • The case of 10 bits, 12 bits or even 14 bits is a bit obscure to me... (dpx 10 bit bvs 12 bits for example) Maybe it was encoded with yuv or such complex things that i would like to avoid dealing with... I am not familiar with 16 bits either (on the coding side) , but we'll see! I am not even sure if short pixels and exr half float are the same thing ?

@janimatic
Copy link
Contributor

@dimitre
i just looked quickly at the new test...
but resizeIOTest_ofImage<float>(thumbnailProxy, "Digital_LAD_HD720_16bit.png");
is using ofImage so the resize is done with FreeImage, not OF if i am not mistaken.
We could try to do the resize though of pixels by loading 16 bits with stb though... (or even simpler, loading float pixels from 16 bits pixels in stb : didn't resizeIOTest_ofPixels<float>(thumbnailProxy, "Digital_LAD_HD720_16bit.png"); work ?)

@dimitre
Copy link
Member

dimitre commented Jul 30, 2024

I think this issue can be tested again with latest master so we know if it can be closed already

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants