Week2_예습과제_윤정

Week2_예습과제_윤정.ipynb

@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"authorship_tag":"ABX9TyMCqyHIPRfJTdNt5OXEOFz0"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["EURON 11기 DL 세션 2주차 예습과제\n","- 4.2.1 활성화 함수 (렐루 + 소프트맥스)\n","- 4.2.1 손실 함수 (MSE, 크로스 엔트로피)\n","- 4.2.3 드롭아웃\n","- 4.2.3 미니 배치 경사 하강법 (Dataset / DataLoader)"],"metadata":{"id":"2iVuWEZWfptB"}},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lQpgS0nFd8I_","executionInfo":{"status":"ok","timestamp":1789365533426,"user_tz":-540,"elapsed":4178,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"cf6459d8-19e8-4945-fd9b-e0cab31073e7"},"outputs":[{"output_type":"stream","name":"stdout","text":["2.11.0+cpu\n"]}],"source":["import torch\n","import torch.nn as nn\n","import torch.nn.functional as F\n","from torch.utils.data import Dataset, DataLoader\n","\n","print(torch.__version__)"]},{"cell_type":"markdown","source":["## 1. 활성화 함수 — 렐루 + 소프트맥스 (p.145)\n","\n","은닉층에는 **렐루(ReLU)**, 출력층에는 **소프트맥스(Softmax)**를 적용한 신경망.\n","\n","- 렐루: 입력이 음수면 0, 양수면 그대로 출력 → 기울기 소멸 문제 없음, 학습 빠름\n","- 소프트맥스: 출력 값을 0~1 사이로 정규화, 총합이 1 → 다중 분류 출력층에 사용\n"],"metadata":{"id":"OdROrt9_f2IZ"}},{"cell_type":"code","source":["class Net(torch.nn.Module):\n","    def __init__(self, n_feature, n_hidden, n_output):\n","        super(Net, self).__init__()\n","        self.hidden = torch.nn.Linear(n_feature, n_hidden)   # 은닉층\n","        self.relu = torch.nn.ReLU(inplace=True)\n","        self.out = torch.nn.Linear(n_hidden, n_output)       # 출력층\n","        self.softmax = torch.nn.Softmax(dim=1)\n","\n","    def forward(self, x):\n","        x = self.hidden(x)\n","        x = self.relu(x)      # 은닉층을 위한 렐루 활성화 함수\n","        x = self.out(x)\n","        x = self.softmax(x)   # 출력층을 위한 소프트맥스 활성화 함수\n","        return x\n","\n","net = Net(n_feature=4, n_hidden=8, n_output=3)\n","sample = torch.randn(2, 4)          # 샘플 2개\n","out = net(sample)\n","print(out)\n","print(\"각 행의 합:\", out.sum(dim=1))  # 소프트맥스라 1에 가까워야 함"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"HpPr0U6wf5Bd","executionInfo":{"status":"ok","timestamp":1789365610095,"user_tz":-540,"elapsed":97,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"1b1a8285-0606-40de-caf9-104d95849172"},"execution_count":2,"outputs":[{"output_type":"stream","name":"stdout","text":["tensor([[0.2479, 0.4134, 0.3387],\n","        [0.2094, 0.3465, 0.4441]], grad_fn=<SoftmaxBackward0>)\n","각 행의 합: tensor([1.0000, 1.0000], grad_fn=<SumBackward1>)\n"]}]},{"cell_type":"markdown","source":["## 2. 손실 함수 ① 평균 제곱 오차 MSE (p.146)\n","\n","MSE = (1/n) Σ (yᵢ − ŷᵢ)²\n","\n","- 실제 값과 예측 값의 차이를 제곱해서 평균\n","- 값이 작을수록 예측력이 좋음\n","- 주로 **회귀**에서 사용\n","\n","> `reduction='sum'` : 평균 대신 합으로 계산. 기본값은 `'mean'`"],"metadata":{"id":"4ctTPWz1gLFH"}},{"cell_type":"code","source":["import torch\n","model=torch.nn.Linear(3,1)\n","x=torch.randn(5,3)\n","y=torch.randn(5,1)\n","loss_fn=torch.nn.MSELoss(reduction='sum')\n","y_pred=model(x)\n","loss=loss_fn(y_pred,y)\n","print(loss)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"d6rLQH-XgNth","executionInfo":{"status":"ok","timestamp":1789365867363,"user_tz":-540,"elapsed":39,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"a5c23810-1388-475e-ca1c-a230d1ab0bb4"},"execution_count":3,"outputs":[{"output_type":"stream","name":"stdout","text":["tensor(4.9158, grad_fn=<MseLossBackward0>)\n"]}]},{"cell_type":"markdown","source":["## 3. 손실 함수 ② 크로스 엔트로피 오차 CEE (p.146~147)\n","\n","CrossEntropy = −Σ yᵢ log ŷᵢ\n","\n","- **분류** 문제에서 원-핫 인코딩했을 때 사용\n","- 시그모이드의 자연 상수 e 때문에 MSE를 쓰면 손실 그래프가 울퉁불퉁 → 로그를 취해 매끈하게 만듦\n","\n","> 코드 설명\n","> - `torch.randn(5, 6)` : 평균 0, 표준편차 1인 정규분포에서 숫자 생성 → 샘플 5개, 클래스 6개짜리 출력\n","> - `torch.empty(5, dtype=torch.long).random_(6)` : 0~5 사이 정수 5개 → 정답 레이블\n","> - `nn.CrossEntropyLoss()`는 내부에서 소프트맥스 + 로그를 같이 처리하므로 원-핫 대신 정수 레이블을 바로 넣음"],"metadata":{"id":"YoYml7XKggRy"}},{"cell_type":"code","source":["loss=nn.CrossEntropyLoss()\n","input=torch.rand(5, 6, requires_grad=True)\n","target=torch.empty(5, dtype=torch.long).random_(6)\n","output=loss(input,target)\n","print(\"target:\", target)\n","print(\"loss:\", output)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"6DsoJeXegqeG","executionInfo":{"status":"ok","timestamp":1789365871245,"user_tz":-540,"elapsed":14,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"f26f3691-1e01-4b7f-dda4-40bf670f84b0"},"execution_count":4,"outputs":[{"output_type":"stream","name":"stdout","text":["target: tensor([5, 5, 2, 1, 3])\n","loss: tensor(1.7444, grad_fn=<NllLossBackward0>)\n"]}]},{"cell_type":"markdown","source":["## 4. 드롭아웃 Dropout (p.150)\n","\n","과적합을 막기 위해 **학습 중 임의로 일부 노드를 제외**하는 방법.\n","\n","- `torch.nn.Dropout(0.5)` : 50%의 노드를 무작위로 골라 사용하지 않음\n","- 784 → 1200 → 1200 → 10 구조 (MNIST 손글씨 28×28=784 픽셀 → 숫자 0~9 분류를 가정)\n","\n","> 드롭아웃은 **학습 때만** 적용되고, 평가 때(`model.eval()`)는 꺼짐"],"metadata":{"id":"rvebzYIAhGrG"}},{"cell_type":"code","source":["class DropoutModel(torch.nn.Module):\n","    def __init__(self):\n","        super(DropoutModel, self).__init__()\n","        self.layer1 = torch.nn.Linear(784, 1200)\n","        self.dropout1 = torch.nn.Dropout(0.5)   # 50%의 노드를 무작위로 선택하여 사용하지 않겠다는 의미\n","        self.layer2 = torch.nn.Linear(1200, 1200)\n","        self.dropout2 = torch.nn.Dropout(0.5)\n","        self.layer3 = torch.nn.Linear(1200, 10)\n","\n","    def forward(self, x):\n","        x = F.relu(self.layer1(x))\n","        x = self.dropout1(x)\n","        x = F.relu(self.layer2(x))\n","        x = self.dropout2(x)\n","        return self.layer3(x)\n","\n","# 동작 확인\n","dropout_model = DropoutModel()\n","sample = torch.randn(1, 784)\n","\n","dropout_model.train()               # 학습 모드 → 드롭아웃 켜짐\n","print(\"train 모드:\", dropout_model(sample)[0, :5])\n","\n","dropout_model.eval()                # 평가 모드 → 드롭아웃 꺼짐\n","print(\"eval 모드 :\", dropout_model(sample)[0, :5])\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"YbHNZ2RDhaLg","executionInfo":{"status":"ok","timestamp":1789367574359,"user_tz":-540,"elapsed":56,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"cfe12b5d-731b-4764-d712-f0b2816eff46"},"execution_count":6,"outputs":[{"output_type":"stream","name":"stdout","text":["train 모드: tensor([-0.1866,  0.0691, -0.2157, -0.0181,  0.2217], grad_fn=<SliceBackward0>)\n","eval 모드 : tensor([-0.1513,  0.0335,  0.0088,  0.1793, -0.0358], grad_fn=<SliceBackward0>)\n"]}]},{"cell_type":"markdown","source":["## 5. 미니 배치 경사 하강법 — Dataset / DataLoader (p.153~154)\n","\n","전체 데이터를 **미니 배치**로 나눠서 배치마다 기울기를 구하고 평균 기울기로 업데이트.\n","배치 경사 하강법보다 빠르고, 확률적 경사 하강법보다 안정적 → 실제로 가장 많이 사용.\n","\n","파이토치에서는 `Dataset`을 상속해 데이터를 정의하고, `DataLoader`로 배치 단위로 꺼냄.\n","\n","- `__len__` : 데이터 개수\n","- `__getitem__` : idx번째 데이터를 텐서로 반환\n","- `batch_size=2` : 미니 배치 크기 (보통 2의 제곱수 사용)\n","- `shuffle=True` : 불러올 때마다 랜덤으로 섞음"],"metadata":{"id":"0hX7gQXTjKRy"}},{"cell_type":"code","source":["class CustomDataset(Dataset):\n","  def __init__(self):\n","        self.x_data = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]\n","        self.y_data = [[12], [18], [11]]\n","  def __len__(self):\n","        return len(self.x_data)\n","  def __getitem__(self, idx):\n","        x = torch.FloatTensor(self.x_data[idx])\n","        y = torch.FloatTensor(self.y_data[idx])\n","        return x, y\n","dataset=CustomDataset()\n","dataloader=DataLoader(\n","    dataset,\n","    batch_size=2,\n","    shuffle=True,\n",")\n","for i, (x_batch, y_batch) in enumerate(dataloader):\n","    print(f\"배치 {i}: x = {x_batch.tolist()}, y = {y_batch.tolist()}\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"WAIVGhv6jP-3","executionInfo":{"status":"ok","timestamp":1789367650799,"user_tz":-540,"elapsed":15,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"354b89db-dcbf-4766-f3ce-04178580eb56"},"execution_count":8,"outputs":[{"output_type":"stream","name":"stdout","text":["배치 0: x = [[1.0, 2.0, 3.0], [7.0, 8.0, 9.0]], y = [[12.0], [11.0]]\n","배치 1: x = [[4.0, 5.0, 6.0]], y = [[18.0]]\n"]}]},{"cell_type":"markdown","source":["## 6. 옵티마이저 선언 모음 (p.155~157)\n","\n","교재 Note에 나온 옵티마이저 선언 코드. `model.parameters()`에 위에서 만든 Net을 넣어 실제로 생성해 봄.\n","\n","| 옵티마이저 | 조정 대상 | lr 기본값 |\n","| --- | --- | --- |\n","| Adagrad | 속도 | 1e-2 |\n","| Adadelta | 속도 | 1.0 |\n","| RMSprop | 속도 | 1e-2 |\n","| SGD + momentum | 운동량 | - |\n","| SGD + nesterov | 운동량 | - |\n","| Adam | 속도 + 운동량 | 1e-3 |"],"metadata":{"id":"fhyq7cg0n5mk"}},{"cell_type":"code","source":["model= Net(n_feature=4, n_hidden=8, n_output=3)\n","\n","optimizer=torch.optim.Adagrad(model.parameters(), lr=0.01)\n","optimizer=torch.optim.Adadelta(model.parameters(), lr=1.0)\n","optimizer=torch.optim.RMSprop(model.parameters(), lr=0.01, momentum=0.9)\n","optimizer=torch.optim.SGD(model.parameters(), lr=0.01)\n","optimizer=torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.9, nesterov=True)\n","optimizer=torch.optim.Adam(model.parameters(), lr=0.01)\n","print(optimizer)\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7NGr3lSsoATu","executionInfo":{"status":"ok","timestamp":1789367907020,"user_tz":-540,"elapsed":5684,"user":{"displayName":"윤정","userId":"14844225516483515025"}},"outputId":"4b89cf93-c16d-499b-b1c6-016c0f885c51"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stdout","text":["Adam (\n","Parameter Group 0\n","    amsgrad: False\n","    betas: (0.9, 0.999)\n","    capturable: False\n","    decoupled_weight_decay: False\n","    differentiable: False\n","    eps: 1e-08\n","    foreach: None\n","    fused: None\n","    lr: 0.01\n","    maximize: False\n","    weight_decay: 0\n",")\n"]}]}]}