CBNU
/
11_Ensemble
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update1

Browse Source
main
hy.kim 1 year ago
parent 99cac452db
commit 718524bf7d
3 changed files with 1111 additions and 0 deletions
Show Stats Download Patch File Download Diff File

192
README.md
Unescape Escape View File

@ -0,0 +1,192 @@
# 앙상블(Enemble)이란?
앙상블(Ensemble) 학습은 여러 개의 머신러닝 모델을 결합하여 단일 모델보다 더 나은 성능을 얻기 위한 기법을 말한다
앙상블 기법은 여러 개의 약한 학습기(weak learner)를 결합하여 강한 학습기(strong learner)를 생성하는 아이디어에 기반한다.
앙상블 기법은 다양한 머신러닝 문제에서 높은 성능을 달성하는 데 매우 효과적이다.
다양한 모델의 특징과 장점을 결합하므로, 오버피팅(과적합)을 줄이고 일반화 성능을 향상시킬 수 있다.
# 앙상블 학습의 주요 기법:
1. **배깅(Bagging, Bootstrap Aggregating)**:
- 동일한 알고리즘에 대해 훈련 데이터의 서로 다른 부분 집합(subset)을 사용하여 여러 모델을 훈련한다.
- 모든 모델의 예측을 집계하여 최종 예측을 생성한다.
- ex) **랜덤 포레스트(Random Forest)**.
2. **부스팅(Boosting)**:
- 연속적으로 모델을 훈련시키면서, 이전 모델의 오류를 다음 모델이 보정한다.
- 모든 모델의 예측을 조합하여 최종 예측을 생성한다.
- ex) **AdaBoost, Gradient Boosting, XGBoost, LightGBM, CatBoost**
3. **스태킹(Stacking)**:
- 여러 다른 모델로부터의 예측을 취합하여, 그 예측들을 입력으로 사용하는 새로운 모델(메타 모델)을 훈련시킨다.
- 이 메타 모델이 최종 예측을 생성한다.
### 배깅(Bagging) 예제 코드[¶]()
<details>
<summary>Code View</summary>
<summary>Bootstrap Aggregating</summary>
<div markdown="1">
````python
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import BaggingClassifier
iris = load_iris()
X, y = iris.data[:, [0,2]], iris.target
model1 = DecisionTreeClassifier(max_depth =10, random_state=0).fit(X, y)
model2 = BaggingClassifier(DecisionTreeClassifier(max_depth=4), n_estimators=50, random_state=0).fit(X, y)
x_min, x_max = X[:,0].min() - 1, X[:,0].max() + 1
y_min, y_max = X[:,1].min() - 1, X[:,1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.1), np.arange(y_min, y_max, 0.1))
plt.subplot(121)
Z1 = model1.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)
plt.contourf(xx, yy, Z1, alpha=0.6, cmap=mpl.cm.jet)
plt.scatter(X[:,0], X[:,1], c=y, alpha=1, s=50, cmap=mpl.cm.jet, edgecolors="k")
plt.title("Decision tree")
plt.subplot(122)
Z2 = model2.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)
plt.contourf(xx, yy, Z2, alpha=0.6, cmap=mpl.cm.jet)
plt.scatter(X[:,0], X[:,1],c=y,alpha=1,s=50,cmap=mpl.cm.jet,edgecolors="k")
plt.title("Bagging of decision trees")
plt.tight_layout()
plt.show()
````
![result](./images/11_2.png)
</div>
<summary>랜덤 포리스트 (Random Forest)</summary>
<div markdown="1">
````python
import pandas as pd
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics
iris = datasets.load_iris()
print('Class names :', iris.target_names)
print('target : [0:setosa, 1:versicolor, 2:virginical]')
print('No. of Data :', len(iris.data))
print('Featrue names :', iris.feature_names)
data = pd.DataFrame({
'sepal length': iris.data[:,0], 'sepal width': iris.data[:,1], 'petal length': iris.data[:,2],
'petal width':iris.data[:,3], 'species':iris.target
})
print(data.head()) # 일부 데이터 출력
x = data[['sepal length', 'sepal width', 'petal length', 'petal width']] # 입력
y = data['species'] # 출력
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3) # 테스트 데이터 30%
print('No. of traing data: ', len(x_train))
print('No. of test data:', len(y_test))
forest = RandomForestClassifier(n_estimators=100) # 모델 생성
forest.fit(x_train, y_train)
y_pred = forest.predict(x_test) # 추론 (예측)
print('Accuracy :', metrics.accuracy_score(y_test, y_pred))
````
</div>
<summary>Result</summary>
<div markdown="1">
````planetext
Class names : ['setosa' 'versicolor' 'virginica']
target : [0:setosa, 1:versicolor, 2:virginical]
No. of Data : 150
Featrue names : ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
sepal length sepal width petal length petal width species
0 5.1 3.5 1.4 0.2 0
1 4.9 3.0 1.4 0.2 0
2 4.7 3.2 1.3 0.2 0
3 4.6 3.1 1.5 0.2 0
4 5.0 3.6 1.4 0.2 0
No. of traing data: 105
No. of test data: 45
Accuracy : 0.9333333333333333
````
</div>
<summary>배깅 회귀 (Bagging Regression)</summary>
<div markdown="1">
````python
import numpy as np
import pandas as pd
from sklearn.datasets import load_boston # scikit-leanr < 1.2
# from sklearn.datasets import fetch_california_housing # replace dataset
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
from sklearn.ensemble import BaggingRegressor
from sklearn.tree import DecisionTreeRegressor
import matplotlib.pyplot as plt
boston = load_boston() # < 1.2
data = pd.DataFrame(boston.data)
data.columns = boston.feature_names
data['PRICE'] = boston.target
print(data.head())
# replace dataset
# california = fetch_california_housing()
# data = pd.DataFrame(california.data)
# data.columns = california.feature_names
# data['PRICE'] = california.target
# print(data.head())
X, y = data.iloc[:,:-1],data.iloc[:,-1]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=123)
bag = BaggingRegressor(base_estimator = DecisionTreeRegressor( ), n_estimators = 10,
max_features=1.0, bootstrap_features=False, random_state=0)
bag.fit(X_train,y_train)
preds = bag.predict(X_test)
rmse = np.sqrt(mean_squared_error(y_test, preds))
print("RMSE: %f" % (rmse))
````
</div>
<summary>Result</summary>
<div markdown="1">
````planetext
CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX
0 0.00632 18.0 2.31 0.0 0.538 6.575 65.2 4.0900 1.0 296.0
1 0.02731 0.0 7.07 0.0 0.469 6.421 78.9 4.9671 2.0 242.0
2 0.02729 0.0 7.07 0.0 0.469 7.185 61.1 4.9671 2.0 242.0
3 0.03237 0.0 2.18 0.0 0.458 6.998 45.8 6.0622 3.0 222.0
4 0.06905 0.0 2.18 0.0 0.458 7.147 54.2 6.0622 3.0 222.0
PTRATIO B LSTAT PRICE
0 15.3 396.90 4.98 24.0
1 17.8 396.90 9.14 21.6
2 17.8 392.83 4.03 34.7
3 18.7 394.63 2.94 33.4
4 18.7 396.90 5.33 36.2
RMSE: 4.594919
````
</div>
</details>

BIN
images/11_2.png
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 62 KiB

919
앙상블과제_2022254026김홍열.ipynb
View File

File diff suppressed because one or more lines are too long
Write Preview
Loading…
Cancel
Save