tl-dr

-Can someone give me step by step instructions (ELI5) on how to get access to my LLM’s on my rig from my phone?

Jan seems the easiest but I’ve tried with Ollama, librechat, etc.

…

I’ve taken steps to secure my data and now I’m going the selfhosting route. I don’t care to become a savant with the technical aspects of this stuff but even the basics are hard to grasp! I’ve been able to install a LLM provider on my rig (Ollama, Librechat, Jan, all of em) and I can successfully get models running on them. BUT what I would LOVE to do is access the LLM’s on my rig from my phone while I’m within proximity. I’ve read that I can do that via wifi or LAN or something like that but I have had absolutely no luck. Jan seems the easiest because all you have to do is something with an API key but I can’t even figure that out.

Any help?

  • tal@lemmy.todayEnglish
    1·
    4 hours ago

    There are 3 lines with the :11434 in them. No brackets or anything like that. -1 has 0.0.0.0 in front -2 has 10.#.#.# in front and has a foreign address that is something other than 0.0.0 -3 is like the 2nd but a slightly different foreign address

    Okay, that…should be okay. As long as all of the addresses that it’s listening on are IPv4 — of the format “x.x.x.x”. No colons in them (other than the colon preceeding “11434”). Not IPv6.

    The subnet mask is 255.255.255.0

    Okay, gotcha. In that case, go ahead with the instructions above, just instead of “/8”, do “/24”. So:

    For “local IP addresses”, you want “These IP Addresses”, and enter 10.0.0.0/24. That’ll be every IPv4 address on your Windows LLM that has “10” as its first number and the first following two numbers the same as yours.

    For “remote IP addresses”, you want “These IP Addresses”, and enter 10.0.0.0/24. Same thing all addresses that start with a “10.” followed by the same following two numbers, which should include your iOS device.

    Oh yes. I’m on windows 10 as well.

    Okay. I think that the interface to add the firewall rule there looks the same as the one I Iinked to above. I went searching for screenshots of adding a hole for a port on Windows 10, and the control panel looks identical to me.

    So, yeah, should be good to go ahead with the above instructions, just using “/24” instead of “/8” in the two places where I mention “/8”. Hopefully after that it’ll be working; if not, then we’ll need to troubleshoot.

    • BlackSnack@lemmy.zipOPEnglish
      1·
      49 minutes ago

      Hate to say it but it didn’t work. I listed below the things I double checked. I really appreciate you helping me troubleshoot this, but it seems like I may have bitten off more than I can chew. I choose Ollama because it was supposed to be one of the easier loca AIs to set up. Do you have any recommendations for alternatives? Or do you think I should incorporate a docker or open web ui as some others have said ?

      -when I went to the ollama app and entered the http://10.#.#.#:11434 , it didn’t work. Also tried the enchanted app and that didn’t work as well.

      -I double checked the rule I made to make sure that was inputted properly. The 10.0.0.0/24 for the local and remote ip addresses.

      -the sanity check went well. The ipv4 said closed. The ipv6 said failed.

      -I checked the netstat -abn thing and 0.0.0.0:11434 is still listening.