Skip to content

Local LLM 적용 #131

Closed
Closed

Description

@jhgoh

Local LLM을 설치해 사용해 보려고 함. naong에 quantized model을 올려 시험 해 볼 수 있을 것 같음.

Activity

  1. added theissue type on Aug 28, 2026
  2. jhgoh commented on Aug 28, 2026

    @jhgoh
    MemberAuthor

    설치 완료. qwen 올려둠.

    jhgoh@naong:~$ !OL
    OLLAMA_MODELS=/srv/ollama/models ollama list
    NAME                       ID              SIZE      MODIFIED       
    nomic-embed-text:latest    0a109f422b47    274 MB    18 minutes ago    
    qwen3:14b                  bdbd181c33f2    9.3 GB    19 minutes ago    
    
    jhgoh@naong:~$ nvidia-smi 
    Fri Aug 28 13:58:51 2026       
    +-----------------------------------------------------------------------------------------+
    | NVIDIA-SMI 580.173.02             Driver Version: 580.173.02     CUDA Version: 13.0     |
    +-----------------------------------------+------------------------+----------------------+
    | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
    |                                         |                        |               MIG M. |
    |=========================================+========================+======================|
    |   0  NVIDIA GeForce GTX 1080 Ti     On  |   00000000:01:00.0 Off |                  N/A |
    | 23%   45C    P2            215W /  220W |    9393MiB /  11264MiB |     99%      Default |
    |                                         |                        |                  N/A |
    +-----------------------------------------+------------------------+----------------------+
    
    +-----------------------------------------------------------------------------------------+
    | Processes:                                                                              |
    |  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
    |        ID   ID                                                               Usage      |
    |=========================================================================================|
    |    0   N/A  N/A            1510      C   ...local/lib/ollama/llama-server       9388MiB |
    +-----------------------------------------------------------------------------------------+
    
  3. jhgoh commented on Aug 28, 2026

    @jhgoh
    MemberAuthor
    jhgoh@lugia:~$ curl -s http://naong:11434/api/generate -d '{"model":"qwen3:14b","prompt":"안녕! 네 소개를 한 문장으로 해줘.","stream":false}'
    {"model":"qwen3:14b","created_at":"2026-08-28T13:59:00.80804235Z","response":"안녕하세요! 저는 Qwen으로, 질문에 답변하고, 콘텐츠를 생성하며, 대화를 도와드릴 수 있는 인공지능 어시스턴트입니다. 다양한 언어를 지원하며, 정확하고 유용한 정보를 제공하기 위해 노력하고 있어요.","thinking":"Okay, the user asked me to introduce myself in one sentence. Let me think about how to do that.\n\nFirst, I need to cover my main functions. I can help with answering questions, creating content, and having conversations. That's a good start.\n\nI should mention the languages I support. The user might be interested in knowing that I can communicate in multiple languages, including Korean. That's important for accessibility.\n\nAlso, I should highlight my ability to provide accurate and helpful information. Users want reliable assistance. Maybe add something about being friendly and approachable to make the tone more welcoming.\n\nWait, the user wrote the query in Korean, so maybe they prefer the response in Korean. But the initial instruction was in Korean, but the answer needs to be in one sentence. Let me check the original request again.\n\nThe user wrote: \"안녕! 네 소개를 한 문장으로 해줘.\" Which means \"Hello! Please introduce yourself in one sentence.\" So they want the response in Korean. But the example response is in English. Hmm, maybe they want the answer in Korean. Let me adjust that.\n\nSo, in Korean, the response would be: \"안녕하세요! 저는 Qwen으로, 질문에 답변하고, 콘텐츠를 생성하며, 대화를 도와드릴 수 있는 인공지능 어시스턴트입니다. 다양한 언어를 지원하며, 정확하고 유용한 정보를 제공하기 위해 노력하고 있어요.\"\n\nWait, but the user might want the response in the same language as the query. Since the query was in Korean, the answer should be in Korean. But the initial example response was in English. Maybe the user is testing if I can switch languages. But the instruction was to introduce myself in one sentence, and the user's message is in Korean. So the answer should be in Korean. Let me make sure.\n\nAlso, I need to keep it concise, one sentence. Let me structure it properly. Start with a greeting, then state my role, mention language support, and my goal. That should cover it without being too long.\n","done":true,"done_reason":"stop","context":[151644,872,198,126246,144370,0,65553,97,142559,18411,61298,53435,40853,42039,60716,144131,13,608,26865,151645,198,151644,77091,198,151667,198,32313,11,279,1196,4588,752,311,19131,7037,304,825,11652,13,6771,752,1744,911,1246,311,653,429,382,5338,11,358,1184,311,3421,847,1887,5746,13,358,646,1492,448,35764,4755,11,6825,2213,11,323,3432,20959,13,2938,594,264,1661,1191,382,40,1265,6286,279,15459,358,1824,13,576,1196,2578,387,8014,304,14063,429,358,646,19032,304,5248,15459,11,2670,16134,13,2938,594,2989,369,39700,382,13394,11,358,1265,11167,847,5726,311,3410,13382,323,10950,1995,13,14627,1366,14720,12994,13,10696,912,2494,911,1660,11657,323,5486,480,311,1281,279,16232,803,35287,382,14190,11,279,1196,6139,279,3239,304,16134,11,773,7196,807,10702,279,2033,304,16134,13,1988,279,2856,7600,572,304,16134,11,714,279,4226,3880,311,387,304,825,11652,13,6771,752,1779,279,4024,1681,1549,382,785,1196,6139,25,330,126246,144370,0,65553,97,142559,18411,61298,53435,40853,42039,60716,144131,1189,15920,3363,330,9707,0,5209,19131,6133,304,825,11652,1189,2055,807,1366,279,2033,304,16134,13,1988,279,3110,2033,374,304,6364,13,88190,11,7196,807,1366,279,4226,304,16134,13,6771,752,7500,429,382,4416,11,304,16134,11,279,2033,1035,387,25,330,126246,144370,91145,0,134561,1207,16948,42039,11,138520,19391,143604,126204,11,3315,121,246,144137,142852,18411,51917,130705,11,60960,56290,18411,129392,80573,29346,135379,28733,64521,58677,78125,21329,66019,124685,134619,94152,28626,78952,13,135392,139957,31079,18411,132185,130705,11,36055,133085,126204,126310,26699,23573,60039,18411,130094,66425,130039,136111,126204,127353,35711,2217,14190,11,714,279,1196,2578,1366,279,2033,304,279,1852,4128,438,279,3239,13,8704,279,3239,572,304,16134,11,279,4226,1265,387,304,16134,13,1988,279,2856,3110,2033,572,304,6364,13,10696,279,1196,374,7497,421,358,646,3398,15459,13,1988,279,7600,572,311,19131,7037,304,825,11652,11,323,279,1196,594,1943,374,304,16134,13,2055,279,4226,1265,387,304,16134,13,6771,752,1281,2704,382,13394,11,358,1184,311,2506,432,63594,11,825,11652,13,6771,752,5944,432,10277,13,5145,448,264,42113,11,1221,1584,847,3476,11,6286,4128,1824,11,323,847,5795,13,2938,1265,3421,432,2041,1660,2238,1293,624,151668,271,126246,144370,91145,0,134561,1207,16948,42039,11,138520,19391,143604,126204,11,3315,121,246,144137,142852,18411,51917,130705,11,60960,56290,18411,129392,80573,29346,135379,28733,64521,58677,78125,21329,66019,124685,134619,94152,28626,78952,13,135392,139957,31079,18411,132185,130705,11,36055,133085,126204,126310,26699,23573,60039,18411,130094,66425,130039,136111,126204,127353,35711,13],"total_duration":18259485764,"load_duration":1413065,"prompt_eval_count":24,"prompt_eval_duration":105410000,"eval_count":501,"eval_duration":18018712000}```
    
  4. jhgoh commented on Aug 28, 2026

    @jhgoh
    MemberAuthor
    Image
  5. jhgoh commented on Aug 28, 2026

    @jhgoh
    MemberAuthor
    Image
  6. jhgoh commented on Aug 28, 2026

    @jhgoh
    MemberAuthor
    Image
  7. jhgoh commented on Sep 27, 2026

    @jhgoh
    MemberAuthor

    잘 실행됨. 어떻게 활용할지는 이후 고민해 볼 것.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    adminServer administration

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions