영상인식 : Keras : Transfer Learning : 기존 모델에 새로운 클래스 학습 : 예제, 실습

깃허브에 존재하는 좋은 코드들을 활용하였습니다. 이번 예제에서는 CIFAR10 클래스를 우선적으로 5개만 훈련을 시킵니다. 그 다음에, 훈련된 모델을 가지고 레이어 마지막 부분을 수정하여 10개의 클래스로 수정해 훈련을 진행합니다. 해당 결과와 코드를 확인해 보겠습니다. 

 

import numpy as np
from matplotlib import pyplot as plt

import keras
from keras.datasets import cifar10
from keras.layers import Dense, Convolution2D, Flatten, Activation, MaxPooling2D, Dropout
from keras.models import Sequential
from keras.utils import np_utils

(x_train, y_train), (x_test, y_test) = cifar10.load_data()
n_examples = 50000


X1_train = []
X1_test = []
X2_train = []
X2_test = []
Y1_train = []
Y1_test = []
Y2_train = []
Y2_test = []

for ix in range(n_examples):
    if y_train[ix] < 5:
        # put data in set 1
        X1_train.append(x_train[ix]/255.0)
        Y1_train.append(y_train[ix])
    else:
        # put data in set 2
        X2_train.append(x_train[ix]/255.0)
        Y2_train.append(y_train[ix])

for ix in range(y_test.shape[0]):
    if y_test[ix] < 5:
        # put data in set 1
        X1_test.append(x_test[ix]/255.0)
        Y1_test.append(y_test[ix])
    else:
        # put data in set 2
        X2_test.append(x_test[ix]/255.0)
        Y2_test.append(y_test[ix])


X1_train = np.asarray(X1_train).reshape((-1, 32, 32, 3))
X1_test = np.asarray(X1_test).reshape((-1, 32, 32, 3))
X2_train = np.asarray(X2_train).reshape((-1, 32, 32, 3))
X2_test = np.asarray(X2_test).reshape((-1, 32, 32, 3))

Y1_train = np_utils.to_categorical(np.asarray(Y1_train), 5)
Y1_test = np_utils.to_categorical(np.asarray(Y1_test), 5)

Y2_train = np_utils.to_categorical(np.asarray(Y2_train), 10)
Y2_test = np_utils.to_categorical(np.asarray(Y2_test), 10)


print (X1_train.shape, X1_test.shape)
print (Y1_train.shape, Y1_test.shape)

split1 = int(0.8 * X1_train.shape[0])
split2 = int(0.8 * X2_train.shape[0])

x1_val = X1_train[split1:]
x1_train = X1_train[:split1]
y1_val = Y1_train[split1:]
y1_train = Y1_train[:split1]

x2_val = X2_train[split2:]
x2_train = X2_train[:split2]
y2_val = Y2_train[split2:]
y2_train = Y2_train[:split2]


model = Sequential()

model.add(Convolution2D(32, 5, 5, input_shape=(32, 32, 3), activation='relu'))
model.add(Convolution2D(16, 5, 5, activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Convolution2D(8, 3, 3, activation='relu'))
model.add(Flatten())
model.add(Dropout(0.42))

model.add(Dense(128))
model.add(Activation('relu'))

model.add(Dense(5))
model.add(Activation('softmax'))

model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])


import time, datetime

start = datetime.datetime.now()
hist1 = model.fit(x1_train, y1_train,
         nb_epoch=10,
         shuffle=True,
         batch_size=100,
         validation_data=(x1_val, y1_val), verbose=2)

time_taken = datetime.datetime.now() - start
print('\n'*2, '-'*20, '\n')
print('Time taken for first training: ', time_taken)
print('\n', '-'*20, '\n'*2)


for l in model.layers[:6]:
    l.trainable = False   

trans_model = Sequential(model.layers[:6])

trans_model.add(Dense(128))
trans_model.add(Activation('relu'))
trans_model.add(Dense(10))
trans_model.add(Activation('softmax'))

trans_model.summary()
trans_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])


start = datetime.datetime.now()
hist2 = trans_model.fit(x2_train, y2_train, nb_epoch=10, shuffle=True, batch_size=100, validation_data=(x2_val, y2_val), verbose=2)
time_taken = datetime.datetime.now() - start
print('\n'*2, '-'*20, '\n' )
print('Time taken for final training: ', time_taken)
print('\n', '-'*20, '\n'*2)

 

결과는 아래와 같습니다.

---

(25000, 32, 32, 3) (5000, 32, 32, 3)
(25000, 5) (5000, 5)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 32)        2432      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 24, 24, 16)        12816     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 12, 12, 16)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 10, 10, 8)         1160      
_________________________________________________________________
flatten_1 (Flatten)          (None, 800)               0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 800)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               102528    
_________________________________________________________________
activation_1 (Activation)    (None, 128)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 5)                 645       
_________________________________________________________________
activation_2 (Activation)    (None, 5)                 0         
=================================================================
Total params: 119,581
Trainable params: 119,581
Non-trainable params: 0
_________________________________________________________________
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:93: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(32, (5, 5), input_shape=(32, 32, 3..., activation="relu")`
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:94: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(16, (5, 5), activation="relu")`
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:96: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(8, (3, 3), activation="relu")`
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:119: UserWarning: The `nb_epoch` argument in `fit` has been renamed `epochs`.
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
 - 72s - loss: 1.2648 - acc: 0.4585 - val_loss: 1.0604 - val_acc: 0.5748
Epoch 2/10
 - 90s - loss: 1.0166 - acc: 0.5779 - val_loss: 0.9817 - val_acc: 0.6054
Epoch 3/10
 - 89s - loss: 0.9583 - acc: 0.6038 - val_loss: 1.0085 - val_acc: 0.6062
Epoch 4/10
 - 98s - loss: 0.9137 - acc: 0.6276 - val_loss: 0.9150 - val_acc: 0.6306
Epoch 5/10
 - 89s - loss: 0.8869 - acc: 0.6419 - val_loss: 0.8580 - val_acc: 0.6596
Epoch 6/10
 - 91s - loss: 0.8546 - acc: 0.6539 - val_loss: 0.8510 - val_acc: 0.6660
Epoch 7/10
 - 97s - loss: 0.8304 - acc: 0.6685 - val_loss: 0.8209 - val_acc: 0.6766
Epoch 8/10
 - 98s - loss: 0.8115 - acc: 0.6746 - val_loss: 0.8015 - val_acc: 0.6886
Epoch 9/10
 - 91s - loss: 0.7846 - acc: 0.6882 - val_loss: 0.7846 - val_acc: 0.6940
Epoch 10/10
 - 93s - loss: 0.7656 - acc: 0.6988 - val_loss: 0.7631 - val_acc: 0.7048


 -------------------- 

Time taken for first training:  0:15:07.559628

 -------------------- 


_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 32)        2432      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 24, 24, 16)        12816     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 12, 12, 16)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 10, 10, 8)         1160      
_________________________________________________________________
flatten_1 (Flatten)          (None, 800)               0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 800)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 128)               102528    
_________________________________________________________________
activation_3 (Activation)    (None, 128)               0         
_________________________________________________________________
dense_4 (Dense)              (None, 10)                1290      
_________________________________________________________________
activation_4 (Activation)    (None, 10)                0         
=================================================================
Total params: 120,226
Trainable params: 103,818
Non-trainable params: 16,408
_________________________________________________________________
C:\ProgramData\Anaconda3\lib\site-packages\ipykernel_launcher.py:148: UserWarning: The `nb_epoch` argument in `fit` has been renamed `epochs`.
Train on 20000 samples, validate on 5000 samples
Epoch 1/10
 - 49s - loss: 0.9355 - acc: 0.6403 - val_loss: 0.7706 - val_acc: 0.7018
Epoch 2/10
 - 40s - loss: 0.7985 - acc: 0.6918 - val_loss: 0.7451 - val_acc: 0.7132
Epoch 3/10
 - 43s - loss: 0.7692 - acc: 0.7083 - val_loss: 0.7234 - val_acc: 0.7166
Epoch 4/10
 - 45s - loss: 0.7362 - acc: 0.7178 - val_loss: 0.7077 - val_acc: 0.7294
Epoch 5/10
 - 40s - loss: 0.7146 - acc: 0.7309 - val_loss: 0.6822 - val_acc: 0.7426
Epoch 6/10
 - 45s - loss: 0.6888 - acc: 0.7398 - val_loss: 0.6660 - val_acc: 0.7482
Epoch 7/10
 - 40s - loss: 0.6708 - acc: 0.7464 - val_loss: 0.6455 - val_acc: 0.7564
Epoch 8/10
 - 47s - loss: 0.6475 - acc: 0.7552 - val_loss: 0.6463 - val_acc: 0.7522
Epoch 9/10
 - 37s - loss: 0.6377 - acc: 0.7599 - val_loss: 0.6396 - val_acc: 0.7608
Epoch 10/10
 - 48s - loss: 0.6177 - acc: 0.7704 - val_loss: 0.6317 - val_acc: 0.7652


 -------------------- 

Time taken for final training:  0:07:15.473708

 -------------------- 

---

 

같은 데이터 양임에도 불구하고, 훈련시간이 절반 가량으로 줄은 것을 확인할 수 있습니다. 따라서 적절하게 훈련된 모델을 활용한다면 훈련 시간을 줄일 수 있다는 것을 알 수 있습니다 .

또한, 이미 학습된 모델을 활용하면 좀 더 높은 정확도를 얻을 수 있는 것도 알 수 있습니다. 따라서 특색이 비슷한 데이터가 훈련된 모델이라면 충분히 학습에 활용하면 좋은 효과를 볼 수 있다는 것입니다. 

이러한 전이학습은 매우 흥미로운 분야이나, 하지만 실제 백테스팅 결과나 다양한 영역에서 활용할 때, 제 생각만큼 동작이 안되는 경우도 있었습니다. 다양한 연구가 진행되고 있고, 전이학습을 활용할 수 있는 도메인에서는 적극 활용해나가는 것도 좋은 방법이지만, 긍정적인 효과가 무조건 작용하지 않을 수도 있다는 것을 알게 되었습니다.