我首先建立了一個包含 13,000 張 512x512 像素 AI 影像的集合。下一步是將這些圖像的元資料載入到資料表中。之後，我會選擇顏色相近的影像來取代放大影像中的部分區域。數據表如下所示：
The H,L,S and R,G,B values match the swatch color. The color should match some color in the sample.The H,L,S and R,G,B values match the swatch color. The color should match some color in the sample.The H,L,S and R,G,B values match the swatch color. The color should match some color in the sample.H、L、S 和 R、G、B 值與色板顏色相符。該顏色應與樣本中的某種顏色相符。
這樣會載入檔名，但不會載入圖片。
imageDir = "\\VBOXSVR\images\"; // ~13,000 512x512 AI generated images
files = Files In Directory( imageDir );
nf = N Items( files );
nfolds = 3; // the images are divided up into 3 sets to make sure the same images don't appear too soon
dt = New Table( "lookup",
Add Rows( nf ),
Set Cell Height( 51 ),
New Column( "filename", Character, setvalues( files ), Set Display Width( 356 ) ),
New Column( "wide" ),
New Column( "tall" ),
New Column( "h_" ), // several nearest neighbor techniques tried;
New Column( "l_" ), // the video uses HLS to find similar colors
New Column( "s_" ),
New Column( "r" ),
New Column( "g" ),
New Column( "b" ),
New Column( "sample", Expression, "None", Set Display Width( 85 ) ),
New Column( "swatch", Expression, "None", Set Display Width( 81 ) ),
New Column( "fold123", formula( Mod( Row(), nfolds ) ) ),
New Column( "nUsed", Numeric, "Nominal", <
<SET each="" value=""></SET>
<P>為了從樣本影像中提取一塊顏色，請使用 HLS 而不是 RGB，這樣可以將相似的色調分組，並將色調分成 32 個不同的色調。載入每張圖像，並將縮小後的圖片放入樣本列（僅用於調試）。在以下程式碼中，`{hue, lite, sat} = Color To HLS( img << getpixels );` 將一個 512x512 的矩陣儲存到 `hue`、`sat` 和 `lite` 變數中。我們將對它們進行逐元素矩陣運算。</P> <P>HLS 的一個問題是，對於包括黑色和白色在內的各種灰色色調，它會忽略色調（H，也就是顏色）部分。我選擇將所有接近灰色的顏色的 H 值設為 1.0，以便將它們歸為一類。如果主要色塊是深黃色、深藍色等，則會被歸類為黑色。</P> <P>將 H、L、S 值介於 0 到 1 之間的區域進行分箱（乘以 nbuckets(32) 並向下取整），然後建立一個名為 total 的數組，其中所有元素均為零。接著，使用 ++ 運算子和一個複雜的下標初始化該陣列。之後，當 LocMax 函數擷取到 total 值最大的陣列的索引時，可以將該索引反向轉換為 H、L、S 值，從而確定該區域對應的顏色。 （最初我以為會有超過 32*32*32 個分箱，空間優化可能更為重要。）</P> <P>while-keeplooking 迴圈嘗試在影像中找到至少有 100 個像素的非灰階顏色；如果找不到，則使用最大的灰階色塊。</P> <CODE class=" language-jsl">nbuckets = 32; // the HLS values are binned into 32 ranges to find similar colors
nbucketsPlus = nbuckets + 1;
For Each Row(
dt,
path = imageDir || dt:filename;
img = New Image( path );
{dt:wide, dt:tall} = img << size;
{hue, lite, sat} = Color To HLS( img << getpixels );
img << scale( .125 );
dt:sample = img; // for debugging, mostly, keep a small copy in the datatable
If( Any( hue >= 1 ),
Show( Row(), Loc( hue >= 1 ) );
Throw( "bad assumptions that hue<1" );
); // use to flag gray scale colors...
// unsaturated is gray; force the hue to 1. Real hues are 0 to .9999
// similar for very low or high lightness
hue[Loc( sat < 0.4 )] = 1; // gray if unsaturated, may be redundant with one of...
hue[Loc( lite < 0.3 )] = 1; // black if low lightness
hue[Loc( lite > 0.7 )] = 1; // white if high lightness
// binning
hue = Floor( hue * nbuckets );//show(hue) 0..8 inclusive
lite = Floor( lite * nbuckets );//show(lite) 0..8 inclusive
sat = Floor( sat * nbuckets );//show(sat) 0..8 inclusive
total = J( nbucketsPlus * nbucketsPlus * nbucketsPlus, 1, 0 );
// HLS is encoded into the subscript for total[]... decoded further below...
total[((nbucketsPlus ^ 2) * hue + nbucketsPlus * lite + sat + 1)[0]]++;
// the HLS chosen for this picture is the largest patch of a
// binned color that is colorful, or gray if no color seems to stand out...
keeplooking = 1;
firstidx = 0;
While( keeplooking,
idxPlus = Loc Max( total ); // a large bin. Often the first one is not a colorful one.
idx = idxPlus - 1;
If( firstidx == 0,
firstidx = idx // But the first one is the gray level if all are non colorful
);
// arbitrarily, there must be at least 100 colorful pixels of the same binned color,
// or use gray...
If( total[idxPlus] < 100, // no color found, use the original gray and the hue will be 1
Write( "\!nOriginal gray" );
dt:h_ = (Floor( firstidx / (nbucketsPlus ^ 2) ) + .5) / nbuckets;
If( dt:h_ < 1,
Throw( "hue?" )
);
firstidx = Mod( firstidx, (nbucketsPlus ^ 2) );
dt:l_ = (Floor( firstidx / (nbucketsPlus) ) + .5) / nbuckets;
firstidx = Mod( firstidx, (nbucketsPlus) );
dt:s_ = (firstidx + .5) / nbuckets;
keeplooking = 0;//
, //
dt:h_ = Floor( idx / (nbucketsPlus ^ 2) ); // decode HLS uses 0-based idx
If( dt:h_ == nbuckets,
//Write( "\!nignore gray" );
total[idxPlus] = 0;// try another until get a color
, //
dt:h_ = (dt:h_ + .5) / nbuckets; // 0 to 1 recovered from binned index
Write( "\!ngot color" );
idx = Mod( idx, (nbucketsPlus ^ 2) );
dt:l_ = (Floor( idx / (nbucketsPlus) ) + .5) / nbuckets;
idx = Mod( idx, (nbucketsPlus) );
dt:s_ = (idx + .5) / nbuckets;
keeplooking = 0;
);
);
);
If( dt:h_ >= 1,
dt:h_ = 1;
dt:s_ = 0;
);
color = HLS Color( dt:h_, dt:l_, dt:s_ );
{dt:r, dt:g, dt:b} = Color To RGB( color ); // later use the RGB equivalent for kdtable nearest neighbor
dt:swatch = New Image( J( 16, 16, color ) ); // for debugging, need to see a swatch of the chosen color
If( Mod( Row(), 10 ) == 0,
dt << gotorow( Row() );
Wait( 0 );
);
);
For Each Row( dt, Row State( dt, Row() ) = Combine States( Marker State( 12 ), Color State( RGB Color( r, g, b ) ) ) );
KDTable 用於在多維空間中尋找最近鄰。這裡我使用了 4 個維度；多出的一個維度會產生三組幾乎總是彼此分離的圖像。
kdlookup = KDTable( dt[0, {r, g, b, fold123}] ); 接下來是一些常數、一些像素數組和一個初始圖像。
VideoRows = 1080;
VideoCols = 1920;
VideoSize = Eval List( List( videocols, videorows ) );
FudgeScale = 1.00001; // fix rounding error in <
<SCALE --="" fractional="" scales="" truncate="" big="" and="" small="" are="" both="" actually="" the="" size="" so="" they="" line="" up="">
1e-14, Throw( "xxx", traceback ) ); upscale = N Rows( BigBitmap ) / nrows; ptemp << scale( FudgeScale * upscale ); {ncols, nrows} = ptemp << size; If( ncols != N Cols( BigBitmap ) | nrows != N Rows( BigBitmap ), Throw( Char( ncols ) || " " || Char( nrows ), traceback ) ); // small is initialized to the start/final image which was just upscaled SmlBitmap[0, 0] = -(ptemp << getpixels); // the normally negative jmp colors are kept as positive in big and small ptemp = 0;//release
</SCALE> 這是一個用於尋找相似位圖的核心函數。最初，「小」點陣圖比「大」點陣圖小得多，但現在它們的大小相同，以保持像素對齊。此函數會建立一個新的大點陣圖，其中包含 512x512 像素的圖像，這些圖像的顏色與小點陣圖相同區域的平均顏色相似。之後，大小位圖將被混合併縮放，以產生視訊幀。
bigFromSml = Function( {fold}, // makes a new big from the 13K 512x512 choices that aligns with small
{irow, icol, patch, red, green, blue, rows, dists, path, img},
For( irow = 0, irow < N Rows( SmlBitmap ), irow += 512,
Show( irow );
Wait( 0 );
srcrow = Floor( irow ) + 1 :: Floor( irow ) + 512;
For( icol = 0, icol < N Cols( SmlBitmap ), icol += 512,
srccol = Floor( icol ) + 1 :: Floor( icol ) + 512;
patch = SmlBitmap[srcrow, srccol]; // carve out a patch of 512x512 pixels from small
blue = Mod( patch, 256 );
green = Mod( patch -= blue, 256 * 256 );
red = Mod( patch -= green, 256 * 256 * 256 );
// 0..255
blue = Mean( blue );
green = Mean( green ) / 256;
red = Mean( red ) / (256 * 256);
// lookup a bitmap with similar color
nchoices = 5; // pick one of these nearest choices
{rows, dists} = kdlookup << KNearestRows( nchoices, red / 255 || green / 255 || blue / 255 || fold );
If( Mean( dists ) > 1.1,
// if the lookup is returning far-away answers, recycle old pictures
Show( kdlookup << insertrows( 1 :: N Rows( dt ) ) )
);
choice = rows[Random Integer( 1, N Items( rows ) )]; // randomly pick one of the 5 best choices
dt:nUsed[choice] += 10 ^ (4 * fold);// debugging
kdlookup << removeRows( choice );// don't reuse this picture until recycled
path = imageDir || dt:filename[choice];
img = Try(
New Image( path )// during development I might delete a file while this is running
, //
Show( path, choice, exception_msg );
choice = 1;
kdlookup << insertrows( choice );
New Image( imageDir || dt:filename[choice] ); // see if row 1 can keep it going
);
{c, r} = img << size;
img << scale( FudgeScale * 512 / c, FudgeScale * 512 / r ); // a few images need fixups
BigBitmap[srcrow, srccol] = -(img << getpixels); // same location for small and big. store positive jmpcolor
);
)
);
fold = 0;
bigFromSml( fold ); // make the original big before the loop, using the small image 在運行影像生成循環之前，還需要設定幾個常數，其中一個是透過調整一些數值計算得出的。 12.8 這個數字一開始有點難理解──它使得縮放速度看起來恆定。隨著混合大小位圖的放大，為了獲得相同的縮放比例，需要移除邊緣的像素越來越少。 12.8 是初始縮放比例與最終縮放比例的比值。
outdir = "\\VBOXSVR\nfsc\deeppic\"; // the video uses a part1 and part2 sequence
// maxrows is the maximum rows that can be removed from the top and bottom without needing to stretch the middle to the VideoRows
maxrows = (N Rows( SmlBitmap ) - VideoRows) / 2;
countmaxrows = 0;
startrows = 48.705; // 48.705 hand picked for 360 frames...6 second transitions
endrows = startrows / (N Rows( SmlBitmap ) / VideoRows);// 12.8 is ratio of start image size to end image size
For( rowReduce = 1, rowReduce <= maxrows, rowReduce += Interpolate( rowReduce, 0, startrows, maxrows, endrows ),
countmaxrows += 1
);
Show( 361, countmaxrows, startrows, endrows );// targeting 180, no, lets use 360 at 60FPS, frames (181-1) (361-1)
最後是生成圖像的循環。這是一個巢狀循環；外層循環呼叫 `bigFromSmall` 函數，為內層循環產生另外 360 幅影像做準備。內層循環從大圖和小圖中提取越來越小的像素塊，對它們進行縮放、混合，並將混合後的圖像寫入磁碟。
// 1e12, etc for naming the output files, this loop is not intended to finish; interupt it when there are enough frames
For( iround = 1e12, iround <= 9e12, iround += 1e6,
Show( iround );
Wait( 0.01 );
pictureNumber = 0;
For( rowReduce = 0, rowReduce <= maxrows, rowReduce += Interpolate( rowReduce, 0, startrows, maxrows, endrows ),
If( rowReduce == 0 & iround != 1e12,
Continue()
);// keep first, later it is an expensive duplicate
colReduce = 16 * rowReduce / 9;
// each image is a blend of small and big; the slice from small and big is combined
// and scaled to the video size. the slices get smaller and smaller and need to be scaled
// down less and less until the middle slice needs a scale of 1:1
start = HP Time();
smlPatch = SmlBitmap[Round( rowReduce ) + 1 :: N Rows( SmlBitmap ) - Round( rowReduce ), Round( colReduce ) + 1 :: N Cols( SmlBitmap ) - Round( colReduce )];
smlImg = New Image( -smlPatch );
{ncols, nrows} = smlImg << size;
smlImg << scale( FudgeScale * VideoCols / ncols, FudgeScale * VideoRows / nrows );
If( smlImg << size != VideoSize,
Throw( Char( smlImg << size ), traceback )
);
{smlred, smlgreen, smlblue} = smlImg << getpixels( "rgb" );
bigPatch = BigBitmap[Round( rowReduce ) + 1 :: N Rows( SmlBitmap ) - Round( rowReduce ), Round( colReduce ) + 1 :: N Cols( SmlBitmap ) - Round( colReduce )];
bigImg = New Image( -bigPatch );
{ncols, nrows} = bigImg << size;
bigImg << scale( FudgeScale * VideoCols / ncols, FudgeScale * VideoRows / nrows );
If( bigImg << size != VideoSize,
Throw( Char( bigImg << size ), traceback )
);
{bigred, biggreen, bigblue} = bigImg << getpixels( "rgb" );
stop = HP Time();
// blend and save the scaled slices
alpha = 1 - (pictureNumber / countmaxrows);
beta = 1 - alpha;
combine = New Image( "rgb", {bigred * beta + smlred * alpha, biggreen * beta + smlgreen * alpha, bigblue * beta + smlblue * alpha} );
combine << savepicture( outdir || Char( iround + (pictureNumber += 1) ) || ".jpg", "jpg" );
Write( "\!n" || Char( (stop - start) / 1e6, 6 ) || " " || Char( iround - 1e12 ) || " " || Char( pictureNumber ) );
Wait( 0.01 );
);
// combine becomes the new smlBitmap
combine << scale( FudgeScale * upscale );
{ncols, nrows} = combine << size;
If( ncols != N Cols( BigBitmap ) | nrows != N Rows( BigBitmap ),
Throw( Char( ncols ) || " " || Char( nrows ), traceback )
);
SmlBitmap[0, 0] = -(combine << getpixels);
combine = 0;//release
// and make a new big bitmap from small
fold = Mod( fold + 1, nfolds );
bigFromSml( fold );
); 圖片創建完成後，Blender 的非線性編輯器可以載入它們並製作影片。我選擇 60 幀/秒而不是更高的空間分辨率，因為上面那些「大」點陣圖已經接近過大的邊緣了。
Blender is open source; I use it on Linux. I think Win and Mac versions are available. It has a long learning curve.Blender is open source; I use it on Linux. I think Win and Mac versions are available. It has a long learning curve.Blender is open source; I use it on Linux. I think Win and Mac versions are available. It has a long learning curve.Blender是開源軟體；我在Linux系統上使用它。我想它應該也有Windows和Mac版本。它的學習曲線比較陡峭。
完成了。我很喜歡這種過渡效果；從選擇顏色到混合，整個過程都很流暢自然。
如果你回到辦公室，開始工作前最好把音量調低一些。
