F650/450、8145、K662等ARM平台，LXC容器内运行OpenWRT子系统
本帖最后由 songee 于 2022-4-27 21:04 编辑家里装宽带，电信200M，送了个中兴F650A。自带的天翼3.0web界面看着不错可惜几乎没有可玩性，闲来无事研究一番。
借鉴论坛前辈achaoge的帖子：中兴 F450A/F650A 免 TTL 持久开 ssh顺利开启telnet，发现该光猫的运作机制是 主系统负责底层的硬件驱动、sendcmd管理系统（telnet、超密、ppp拨号、网络及WiFi、ftp等）、以httpd为基础的8080端口电信web管理界面，最后启动saf（lxc容器管理）程序。
saf程序在华为和中兴的很多光猫上都存在，根据该程序在F650A中的功能，我认为其主要作用是自动管理lxc容器。开机有 ‘saf service 6 7 11’进程，意为将 mtd6 或 mtd7 作为saf子系统的firmware（rootfs）分区，mtd11 作为APPS数据overlay分区。至于是 mtd6 还是 mtd7，由 saf setactive 命令指定，默认 mtd6，如果子系统启动失败自动指定 mtd7；从 mtd7 启动失败可能导致整个 mtd11 分区被擦除，mtd11 分区同时也是主系统的overlay数据存放区，操作时需要特别注意！！
root@F650A:/usr/sbin # saf
Usage: saf service FWK_MTD_ID1 FWK_MTD_ID2 APPS_MTD_ID (eg. saf service 9 10 7)
saf console
saf info
saf version
saf getfwkver
saf getbakfwkver
saf getactive
saf setactive FWK_MTD_ID (eg. ./saf setactive 17 1)root@F650A:~ # cat /proc/mtd
dev: size erasesizename
mtd0: 08000000 00020000 "whole flash"
mtd1: 00100000 00020000 "u-boot"
mtd2: 00080000 00020000 "others"
mtd3: 00100000 00020000 "parameter tags"
mtd4: 00100000 00020000 "wlan"
mtd5: 00200000 00020000 "usercfg"
mtd6: 00800000 00020000 "framework"
mtd7: 00800000 00020000 "framework1"
mtd8: 00780000 00020000 "dbus"
mtd9: 01280000 00020000 "kernel1"
mtd10: 01280000 00020000 "kernel2"
mtd11: 03e00000 00020000 "plugin_data"
mtd12: 01240000 00020000 "rootfs"
saf程序管理子系统，是通过系统的lxc容器实现的，在/dbus/usr/bin/目录下有lxc相关程序，但并非标准的官方版，而是定制的，lxc.lxcpath即lxc容器存放位置设定为/usr/var/lib/lxc，暂时没有发现更改的方法（也没什么必要）。root@F650A:~ # ls -l /dbus/usr/bin
total 377
-rwxr-xr-x 1 root root 10143 Nov 152018 dbus-cleanup-sockets
-rwxr-xr-x 1 root root 18867 Nov 152018 dbus-launch
-rwxr-xr-x 1 root root 22537 Nov 152018 dbus-monitor
-rwxr-xr-x 1 root root 11049 Nov 152018 dbus-run-session
-rwxr-xr-x 1 root root 23431 Nov 152018 dbus-send
-rwxr-xr-x 1 root root 19495 Nov 152018 dbus-test-tool
-rwxr-xr-x 1 root root 12257 Nov 152018 dbus-update-activation-environment
-rwxr-xr-x 1 root root 8365 Nov 152018 dbus-uuidgen
-rwxr-xr-x 1 root root 33112 Nov 152018 gdbus
-rwxrwxrwx 1 root root 12332 Nov 152018 lxc-attach
-rwxrwxrwx 1 root root 14241 Nov 152018 lxc-autostart
-rwxrwxrwx 1 root root 9122 Nov 152018 lxc-cgroup
-rwxrwxrwx 1 root root 4414 Nov 152018 lxc-checkconfig
-rwxrwxrwx 1 root root 10175 Nov 152018 lxc-clone
-rwxrwxrwx 1 root root 6181 Nov 152018 lxc-config
-rwxrwxrwx 1 root root 8590 Nov 152018 lxc-console
-rwxrwxrwx 1 root root 14556 Nov 152018 lxc-create
-rwxrwxrwx 1 root root 8540 Nov 152018 lxc-destroy
-rwxrwxrwx 1 root root 10027 Nov 152018 lxc-execute
-rwxrwxrwx 1 root root 8630 Nov 152018 lxc-freeze
-rwxrwxrwx 1 root root 18956 Nov 152018 lxc-info
-rwxrwxrwx 1 root root 2834 Nov 152018 lxc-ls
-rwxrwxrwx 1 root root 11122 Nov 152018 lxc-monitor
-rwxrwxrwx 1 root root 12270 Nov 152018 lxc-snapshot
-rwxrwxrwx 1 root root 15356 Nov 152018 lxc-start
-rwxrwxrwx 1 root root 10055 Nov 152018 lxc-stop
-rwxrwxrwx 1 root root 8634 Nov 152018 lxc-unfreeze
-rwxrwxrwx 1 root root 12380 Nov 152018 lxc-unshare
-rwxrwxrwx 1 root root 13324 Nov 152018 lxc-usernsexec
-rwxrwxrwx 1 root root 8561 Nov 152018 lxc-wait
root@F650A:~ # lxc-config -l
lxc.default_config
lxc.lxcpath
lxc.bdev.lvm.vg
lxc.bdev.lvm.thin_pool
lxc.bdev.zfs.root
root@F650A:~ # lxc-config lxc.default_config
/usr/etc/lxc/default.conf
root@F650A:~ # lxc-config lxc.lxcpath
/usr/var/lib/lxc
了解系统运作机制之后，我最初打算借用saf方式管理容器，将 mtd6 分区用 'dd' 命令输出到U盘，Ubuntu下'unsquashfs'，然后用官方版openwrt中的 /bin、/sbin、/usr 、/lib 等替换 mtd6 中rootfs下相应文件后 'mksqushfs' 再回光猫 'mtd write' ，但只在15.05.1版本中成功，怀疑是因为从17.01版本开始，openwrt的libc用 musl 取代了15.05及之前的 uClibc，导致saf运行所需的appmgr等程序无法正常运行。鉴于此，索性完全抛弃saf，以lxc方式开启新世界，操作步骤在本文回复中细谈。
经过几番折腾，已将系统原有的openwrt中兴定制版saf子系统完全替换为官方openwrt19.07.1（lxc方式启动），不影响光猫拨号且可在线安装及更新非kmod类app，如ariang离线下载，samba文件共享，甚至各类科学上网工具（性能及安全原因不推荐）。
简化并更新OpenWRT22.03.0 RC1版本
本帖最后由 songee 于 2022-4-26 23:39 编辑因二楼原帖过程过于繁琐，近期尝试了一种较为简单的办法。
1、下载openwrt.org固件，我选的是openwrt-22.03.0-rc1-bcm53xx-generic-tenda_ac9-squashfs.trx，binwalk分析squashfs偏移量并用dd取出，unsquashfs提取rootfs，打包为rootfs2203rc1.tar.gz，传入光猫临时目录，如 /run/shm 。
2、将rootfs2203rc1.tar.gz提取至 (lxc-config lxc.lxcpath)/openwrt/rootfs 。
root@F650A:/usr/var/lib/lxc/openwrt/rootfs # tar -xzvf /run/shm/rootfs2203rc1.tar.gz -C ./
3、修改rootfs。openwrt的网络配置与主系统有冲突，需要清除。
3.1 移除 ./etc/board.d/ 目录下 01_leds 、02_network 、99-default_network 三个文件。
3.2 移除 ./etc/rc.d/ 目录下 K21wpad 、K85odhcpd 、K90network 、S19dnsmasq 、S19firewall 、S19wpad 、S20network 、S35odhcpd 、S96led。
3.3 移除 ./etc/modules.d/ 和 ./etc/modules-boot.d/ 及 ./lib/modules/5.10.111/目录下所有文件，openwrt的kernel modules与光猫内核不兼容。
3.4 修改opkg源 ./etc/opkg/distfeeds.conf（可选）
src/gz openwrt_core https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/targets/bcm53xx/generic/packages
src/gz openwrt_base https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/base
src/gz openwrt_luci https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/luci
src/gz openwrt_packages https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/packages
src/gz openwrt_routing https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/routing
src/gz openwrt_telephony https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/telephony
3.5 修改 ./etc/inittab 实现启动项自启（参考https://github.com/mikma/lxd-openwrt/blob/master/files/etc/inittab）
::sysinit:/etc/init.d/rcS S boot
::shutdown:/etc/init.d/rcS K shutdown
console::askfirst:/usr/libexec/login.sh 3.6 修改密码 （ root 密码 chinadsl ）
root@F650A:/usr/var/lib/lxc/openwrt/rootfs # chroot ./
BusyBox v1.35.0 (2022-04-20 10:37:37 UTC) built-in shell (ash)
root@F650A:/ # passwd
Changing password for root
New password:
Bad password: too weak
Retype password:
passwd: password for root changed by root 3.7 删除 ./etc/resolv.conf 软连接并新建./etc/resolv.conf，否则openwrt无法解析域名
nameserver 223.5.5.5
nameserver 2001:da8::666
4、修改lxc配置文件 /usr/var/lib/lxc/openwrt/config ，根据各自光猫环境修改测试，这个配置我的F650A可正常使用。
lxc.arch = arm
lxc.utsname = openwrt
#lxc.rootfs.mount = /usr/var/lib/lxc/openwrt/rootfs
#lxc.rootfs.path = overlayfs:/opt/upt/framework1/rootfs:/opt/upt/apps/openwrt-rootfs
lxc.rootfs = /usr/var/lib/lxc/openwrt/rootfs
lxc.haltsignal = SIGUSR1
lxc.tty = 1
lxc.pts = 1
#lxc.hook.pre-start = /usr/var/lib/lxc/openwrt/pre-start.sh
lxc.aa_profile = lxc-container-default-with-nesting
lxc.network.type = none
lxc.mount.auto = cgroup-full:rw
#lxc.mount.auto = proc sys cgroup
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.entry = proc proc proc defaults 0 0
5、启动，根据提示按enter进入控制台。
root@F650A:~ # lxc-start -n openwrt
<6>init: Console is alive
<3>init: Failed to redirect stdin to /dev/null: No such file or directory
<3>init: Failed to redirect stdout to /dev/null: No such file or directory
<3>init: Failed to redirect stderr to /dev/null: No such file or directory
<6>kmodloader: loading kernel modules from /etc/modules-boot.d/*
<6>kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<6>init: - preinit -
ls: /etc/board.d/*: No such file or directory
Press the key and hit to enter failsafe mode
Press the , , or key and hit to select the debug level
<3>mount_root: Could not open mtd device: /dev/mtd12
<3>mount_root: reading rootfs failed
<5>mount_root: mounting /dev/root
urandom-seed: Something is wrong with /dev/urandom
<6>procd: - early -
<6>procd: - ubus -
<6>procd: - init -
Please press Enter to activate this console.
<6>kmodloader: loading kernel modules from /etc/modules.d/*
<6>kmodloader: done loading kernel modules from /etc/modules.d/*
<6>urngd: v1.0.2 started.
BusyBox v1.35.0 (2022-04-20 10:37:37 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.||||.----.||_
| - ||_|-__| |||||| _|| _|
|_______|| __|_____|__|__||________||__||____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt 22.03.0-rc1, r19302-df622768da
-----------------------------------------------------
root@openwrt:/#
root@openwrt:/# ps
PID USER VSZ STAT COMMAND
1 root 1452 S /sbin/procd
58 ubus 1112 S /sbin/ubusd
59 root 1196 S /bin/ash --login
93 root 908 S /sbin/urngd
204 root 3736 S /usr/sbin/uhttpd -f -h /www -r OpenWrt -x /cgi-bin -u /ubus -t 60 -T 30
487 root 1952 S /sbin/rpcd -s /var/run/ubus/ubus.sock -t 30
556 root 956 S /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p 22 -K 300 -T 3
1065 root 1200 R ps
root@openwrt:/#
root@openwrt:/# opkg update
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/targets/bcm53xx/generic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_core
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/targets/bcm53xx/generic/packages/Packages.sig
Signature check passed.
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/base/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_base
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/base/Packages.sig
Signature check passed.
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_luci
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/luci/Packages.sig
Signature check passed.
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_packages
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/packages/Packages.sig
Signature check passed.
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_routing
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/routing/Packages.sig
Signature check passed.
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading https://mirrors.tuna.tsinghua.edu.cn/openwrt/releases/22.03.0-rc1/packages/arm_cortex-a9/telephony/Packages.sig
Signature check passed.
root@openwrt:/#
此时临时的openwrt系统已建好，运行的服务有两个：http占用主系统的80和443端口，ssh占用22端口。
如果光猫有公网IP，请务必关闭ssh密码登录并使用密钥。
后期使用建议使用 lxc-start -dn openwrt 后台启动，使用 lxc-stop -kn openwrt 命令关闭。openwrt管理使用ssh密钥登录。
需要固化openwrt到固件的，可以打包rootfs和config（需修改配置），mksquashfs制作成squashfs，用mtd写入saf分区，就不用担心恢复出厂设置了。
随光猫启动可以参考二楼。
因光猫原系统的busybox功能不完整，也没有nano和dropbear，调试lxc可能会不方便，现分享出来。
root@F650A:~ # /usr/bin/busybox
BusyBox v1.23.2 (2020-06-16 16:53:26 CST) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2012.
Licensed under GPLv2. See source distribution for detailed
copyright notices.
Usage: busybox ...]
or: busybox --list[-full]
or: busybox --install [-s]
or: function ...
BusyBox is a multi-call binary that combines many common Unix
utilities into a single executable.Most people will create a
link to busybox for each function they wish to use and BusyBox
will act like whatever it was invoked as.
Currently defined functions:
[, [[, addgroup, adduser, ar, arp, arping, ash, awk, basename, blkid, brctl, bunzip2,
bzcat, cat, catv, chattr, chgrp, chmod, chown, chroot, chrt, chvt, cksum, clear, cmp, cp,
cpio, crond, crontab, cut, date, dc, dd, deallocvt, delgroup, deluser, depmod, devmem, df,
diff, dirname, dmesg, dnsd, dnsdomainname, dos2unix, du, dumpkmap, echo, egrep, eject, env,
ether-wake, expr, false, fbset, fdflush, fdformat, fdisk, fgrep, find, flock, fold, free,
freeramdisk, fsck, fstrim, fuser, getopt, getty, grep, gunzip, gzip, halt, hdparm, head,
hexdump, hostid, hostname, hwclock, id, ifconfig, ifdown, ifup, inetd, init, insmod,
install, ip, ipaddr, ipcrm, ipcs, iplink, iproute, iprule, iptunnel, kill, killall,
killall5, klogd, last, less, linux32, linux64, linuxrc, ln, loadfont, loadkmap, logger,
login, logname, losetup, ls, lsattr, lsmod, lsof, lspci, lsusb, lzcat, lzma, makedevs,
md5sum, mdev, mesg, microcom, mkdir, mke2fs, mkfifo, mkfs.ext2, mknod, mkswap, mktemp,
modinfo, modprobe, more, mount, mountpoint, mt, mv, nameif, nanddump, nandwrite, netstat,
nice, nohup, nslookup, od, openvt, passwd, patch, pidof, ping, pipe_progress, pivot_root,
poweroff, printenv, printf, ps, pwd, rdate, readlink, readprofile, realpath, reboot,
renice, reset, resize, rm, rmdir, rmmod, route, run-parts, runlevel, sed, seq, setarch,
setconsole, setkeycodes, setlogcons, setserial, setsid, sh, sha1sum, sha256sum, sha3sum,
sha512sum, sleep, sort, start-stop-daemon, strings, stty, su, sulogin, swapoff, swapon,
switch_root, sync, sysctl, syslogd, tail, tar, tee, telnet, telnetd, test, tftp, time, top,
touch, tr, traceroute, traceroute6, true, tty, ubiattach, ubidetach, ubimkvol, ubirmvol,
ubirsvol, ubiupdatevol, udhcpc, umount, uname, uniq, unix2dos, unlink, unlzma, unxz, unzip,
uptime, usleep, uudecode, uuencode, vconfig, vi, vlock, watch, watchdog, wc, wget, which,
who, whoami, xargs, xz, xzcat, yes, zcat
root@F650A:~ # nano -h
Usage: nano [[+LINE,COLUMN] FILE]...
Option GNU long option Meaning
+LINE,COLUMN Start at line LINE, column COLUMN
-D --boldtext Use bold instead of reverse video text
-K --rebindkeypad Fix numeric keypad key confusion problem
-L --nonewlines Don't add newlines to the ends of files
-O --morespace Use one more line for editing
-R --restricted Restricted mode
-T <#cols> --tabsize=<#cols> Set width of a tab to #cols columns
-V --version Print version information and exit
-c --const Constantly show cursor position
-d --rebinddelete Fix Backspace/Delete confusion problem
-h --help Show this help text
-l --nofollow Don't follow symbolic links, overwrite
-n --noread Do not read the file (only write it)
-p --preserve Preserve XON (^Q) and XOFF (^S) keys
-t --tempfile Auto save on exit, don't prompt
-v --view View mode (read-only)
-x --nohelp Don't show the two help lines
-z --suspend Enable suspension
root@F650A:~ # nano -V
GNU nano, version 2.4.2
(C) 1999..2015 Free Software Foundation, Inc.
Email: nano@nano-editor.org Web: http://www.nano-editor.org/
Compiled options: --enable-tiny --disable-utf8
root@F650A:~ # dropbear -h
Dropbear server v2015.67 https://matt.ucc.asn.au/dropbear/dropbear.html
Usage: dropbear
-b bannerfile Display the contents of bannerfile before user login
(default: none)
-r keyfileSpecify hostkeys (repeatable)
defaults:
dss /etc/dropbear/dropbear_dss_host_key
rsa /etc/dropbear/dropbear_rsa_host_key
ecdsa /etc/dropbear/dropbear_ecdsa_host_key
-R Create hostkeys as required
-F Don't fork into background
-E Log to stderr rather than syslog
-m Don't display the motd on login
-w Disallow root logins
-s Disable password logins
-g Disable password logins for root
-B Allow blank password logins
-j Disable local port forwarding
-k Disable remote port forwarding
-a Allow connections to forwarded ports from any host
-p port
Listen on specified tcp port (and optionally address),
up to 10 can be specified
(default port is 22 if none specified)
-P PidFile Create pid file PidFile
(default /var/run/dropbear.pid)
-i Start for inetd
-W <receive_window_buffer> (default 24576, larger may be faster, max 1MB)
-K <keepalive>(0 is never, default 0, in seconds)
-I <idle_timeout>(0 is never, default 0, in seconds)
-V Version
本帖最后由 btworm 于 2021-11-27 21:41 编辑
songee 发表于 2021-11-27 04:15
1.只要lxc相关命令可以正常运行，写好配置文件，选择符合CPU架构的rootfs，就可以把容器跑起来。
2.我的 ...
感谢指点!
关于第2点
华为光猫 / 只读，/etc,/usr,/lib这些目录也都依附于/, 所以都不可写, /opt目录不存在，当然也无法创建。
之前担心/dev/pts, /proc, /sys只读，是因为以前玩路由器时，有个chroot脚本使用过这些目录，刚又看了遍那个脚本，使用的是 "-o bind"参数，所以主机目录是否可写就无所谓了。在华为猫上跑lxc应该没问题了
另外想问问，光猫容器内的应用对端口的监听是：
1.直接监听光猫的端口？
2.还是只能监听容器内的端口，并需要光猫iptables做端口转发给容器？ anysoft 发表于 2020-2-22 17:50
HS8145V HS8145V5F650A 都是512MB+256MB 2.4G+5G方案
价格都还好，特别F650A 因为会玩的人少价格特别便 ...
hs8145v技术成熟，更适合玩，而且其实f650a是512/128，nand flash少了一半，CPU一个尿性，价格差别不大的话，能hs8145v还是别选650。
还有个大胆想法，lxc可以通过配置文件config修改子系统网络，从eth0桥接是不是就能在子系统多拨了，子系统专门离线和网络存储，不影响猫后面接的手机电脑上网，还是蛮有趣的
二楼原帖，仅保留做参考
本帖最后由 songee 于 2022-4-27 21:17 编辑（2020年原帖初步摸索，过程过于繁杂，不建议再按这部分思路操作）
中兴、华为等带有saf模块的光猫，以官方版openwrt替换厂商定制版的思路总结一下，供大家参考。
1、准备工具
F650A主系统没有 mtd 、 dd 和 vi 等工具，而自带的saf子系统正好都有，子系统位于framework分区即mtd6，挂载于/opt/upt/framework，cp过来可以直接使用。root@F650A:~ # cp /opt/upt/framework/saf/rootfs/sbin/mtd /usr/bin/
root@F650A:~ # cp /opt/upt/framework/saf/rootfs/lib/libubox.so /lib/
root@F650A:~ # cp /opt/upt/framework/saf/rootfs/bin/busybox /usr/bin/
root@F650A:~ # cd /usr/bin/
root@F650A:/usr/bin # ln -s busybox dd
root@F650A:/usr/bin # ln -s busybox vi
2、改进lxd-openwrt生成代码。
本节参考https://github.com/mikma/lxd-openwrt,系统推荐Ubuntu 18.04以上。目前该项目支持x86_64|i686|aarch64架构，而F650A的CPU型号 ZTE ZX279128 的架构是 ARMv7 Processor rev 1 (v7l) ，不属于以上三种，只能调整build.sh的部分代码。ZX279128架构与bcm53xx的相同，借用它的SDK。可以git clone 或者网页内 Download ZIP，我浏览器挂的有代理，网页下载之后解压，修改build.sh。修改后的build.sh与原文件build.sh.origin的diff如下
*** build.sh 2020-02-19 01:50:10.972675562 -0100
--- build.sh.origin 2020-02-14 19:10:45.000000000 -0100
***************
*** 72,81 ****
subarch=64
arch_ipk=x86_64
;;
! arm)
! arch=bcm53xx
! subarch=generic
! arch_ipk=arm_cortex-a9
;;
*)
usage
--- 72,81 ----
subarch=64
arch_ipk=x86_64
;;
! aarch64)
! arch=armvirt
! subarch=64
! arch_ipk=aarch64_generic
;;
*)
usage
***************
*** 87,97 ****
if test $ver = snapshot; then
openwrt_branch=snapshot
procd_url=https://github.com/openwrt/openwrt/trunk/package/system/procd
! openwrt_url=https://mirrors.cloud.tencent.com/lede/snapshots/targets/${arch}/${subarch}
else
openwrt_branch=${dist}-${branch_ver}
procd_url=https://github.com/openwrt/openwrt/branches/${openwrt_branch}/package/system/procd
! openwrt_url=https://mirrors.cloud.tencent.com/lede/releases/${ver}/targets/${arch}/${subarch}
fi
procd_extra_ver=lxd-3
--- 87,97 ----
if test $ver = snapshot; then
openwrt_branch=snapshot
procd_url=https://github.com/openwrt/openwrt/trunk/package/system/procd
! openwrt_url=https://downloads.openwrt.org/snapshots/targets/${arch}/${subarch}
else
openwrt_branch=${dist}-${branch_ver}
procd_url=https://github.com/openwrt/openwrt/branches/${openwrt_branch}/package/system/procd
! openwrt_url=https://downloads.openwrt.org/releases/${ver}/targets/${arch}/${subarch}
fi
procd_extra_ver=lxd-3
***************
*** 112,125 ****
}
download_rootfs() {
! # detect_url "rootfs\.tar"
! # local rootfs_url=$openwrt_url/$return
# global $rootfs
! rootfs=dl/openwrt-19.07.1-bcm53xx-generic-rootfs.tar.gz
! # download $rootfs_url $rootfs $rootfs
! # check $rootfs $rootfs_url
}
download_sdk() {
--- 112,125 ----
}
download_rootfs() {
! detect_url "rootfs\.tar"
! local rootfs_url=$openwrt_url/$return
# global $rootfs
! rootfs=dl/$(basename $rootfs_url)
! download $rootfs_url $rootfs
! check $rootfs $rootfs_url
}
download_sdk() {
3、从腾讯镜像下载openwrt-19.07.1-bcm53xx-tplink-archer-c9-v1-squashfs.bin，从固件中提取rootfs并预处理。
x@x-Surface-Pro-3:~/下载$ binwalk openwrt-19.07.1-bcm53xx-tplink-archer-c9-v1-squashfs.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
8317 0x207D TRX firmware header, little endian, image size: 1810432 bytes, CRC32: 0xBCD36218, flags: 0x0, version: 1, header size: 28 bytes, loader offset: 0x1C, linux kernel offset: 0x0, rootfs offset: 0x0
8345 0x2099 LZMA compressed data, properties: 0x5D, dictionary size: 65536 bytes, uncompressed size: 1801662 bytes
1818749 0x1BC07D Squashfs filesystem, little endian, version 4.0, compression:xz, size: 1856770 bytes, 1174 inodes, blocksize: 262144 bytes, created: 2020-01-29 16:05:35
x@x-Surface-Pro-3:~/下载$ dd if=openwrt-19.07.1-bcm53xx-tplink-archer-c9-v1-squashfs.bin of=rootfs-squashfs.bin bs=1 skip=1818749
记录了1900548+0 的读入
记录了1900548+0 的写出
1900548 bytes (1.9 MB, 1.8 MiB) copied, 5.16823 s, 368 kB/s
x@x-Surface-Pro-3:~/下载$ binwalk rootfs-squashfs.bin
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 Squashfs filesystem, little endian, version 4.0, compression:xz, size: 1856770 bytes, 1174 inodes, blocksize: 262144 bytes, created: 2020-01-29 16:05:35
x@x-Surface-Pro-3:~/下载$ unsquashfs rootfs-squashfs.bin
Parallel unsquashfs: Using 4 processors
1071 inodes (1066 blocks) to write
create_inode: could not create character device squashfs-root/dev/console, because you're not superuser!
[====================================================================================================| ] 1065/106699%
created 875 files
created 103 directories
created 195 symlinks
created 0 devices
created 0 fifos
x@x-Surface-Pro-3:~/下载$ cd ./squashfs-root/
x@x-Surface-Pro-3:~/下载/squashfs-root$ gedit etc/opkg/distfeeds.conf将opkg软件源改为腾讯镜像
src/gz openwrt_core https://mirrors.cloud.tencent.co ... xx/generic/packages
src/gz openwrt_kmods https://mirrors.cloud.tencent.co ... 92b7d7eca715b1ee0b8
src/gz openwrt_base https://mirrors.cloud.tencent.co ... /arm_cortex-a9/base
src/gz openwrt_luci https://mirrors.cloud.tencent.co ... /arm_cortex-a9/luci
src/gz openwrt_packages https://mirrors.cloud.tencent.co ... _cortex-a9/packages
src/gz openwrt_routing https://mirrors.cloud.tencent.co ... m_cortex-a9/routing
src/gz openwrt_telephony https://mirrors.cloud.tencent.co ... cortex-a9/telephony
x@x-Surface-Pro-3:~/下载/squashfs-root$ tar czf ../openwrt-19.07.1-bcm53xx-generic-rootfs.tar.gz ./
x@x-Surface-Pro-3:~/下载/squashfs-root$ cd ../
x@x-Surface-Pro-3:~/下载$ mkdir lxd-openwrt-master/dl
x@x-Surface-Pro-3:~/下载$ mv openwrt-19.07.1-bcm53xx-generic-rootfs.tar.gz lxd-openwrt-master/dl/执行修改过的build.sh
x@x-Surface-Pro-3:~/下载$ cd lxd-openwrt-master/
x@x-Surface-Pro-3:~/下载/lxd-openwrt-master$ ./build.sh -a arm -v 19.07.1
......
Pack rootfs
Tarball built: bin/openwrt-19.07.1-bcm53xx-generic-lxd.tar.gz
x@x-Surface-Pro-3:~/下载/lxd-openwrt-master$ mkdir ../openwrt-19.07.1-bcm53xx-generic-lxc
x@x-Surface-Pro-3:~/下载/lxd-openwrt-master$ tar zxf bin/openwrt-19.07.1-bcm53xx-generic-lxd.tar.gz -C ../openwrt-19.07.1-bcm53xx-generic-lxc
x@x-Surface-Pro-3:~/下载/lxd-openwrt-master$ cd ../openwrt-19.07.1-bcm53xx-generic-lxc/
x@x-Surface-Pro-3:~/下载/openwrt-19.07.1-bcm53xx-generic-lxc$ ls
metadata.yamlrootfstemplates
x@x-Surface-Pro-3:~/下载/openwrt-19.07.1-bcm53xx-generic-lxc$
4、这时我们已经创建好lxc下可运行的openwrt19.07.1的rootfs,接下来处理lxc及rootfs的配置文件。
进光猫复制自带子系统的两个文件，再通过U盘或ftp传到Ubuntu的openwrt-19.07.1-bcm53xx-generic-lxc目录下，
root@F650A:~ # cp /opt/upt/framework/saf/config /mnt/USB_disc1/
root@F650A:~ # cp /opt/upt/framework/pre-start.sh /mnt/USB_disc1/config为saf系统的配置文件，新系统下稍作修改,
lxc.arch = arm
lxc.utsname = openwrt
lxc.rootfs.mount = /usr/var/lib/lxc/openwrt/rootfs
lxc.rootfs.path = overlayfs:/opt/upt/framework1/rootfs:/opt/upt/apps/openwrt-rootfs
# lxc.rootfs = /usr/var/lib/lxc/openwrt/rootfs
lxc.haltsignal = SIGUSR1
lxc.tty = 1
lxc.pts = 1
lxc.hook.pre-start = /usr/var/lib/lxc/openwrt/pre-start.sh
lxc.aa_profile = lxc-container-default-with-nesting
lxc.network.type = none
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.auto = cgroup-full:rw
lxc.mount.entry = proc proc proc defaults 0 0
lxc.mount.entry = /mnt mnt none rw,rbind 0 0
lxc.start.auto = 1
pre-start.sh处理模拟终端，为避免重复挂载devpts，添加一行卸载代码，
#!/bin/sh
umount /dev/pts /dev/ptmx
mount -t devpts -o newinstance -o ptmxmode=0666,nosuid devpts /dev/pts
mount -o bind /dev/pts/ptmx /dev/ptmx
再删除rootfs/etc目录下网络和防火墙相关启动及配置文件如uci-defaults/09_fix_crc、uci-defaults/12_network-generate-ula、uci-defaults/70_fill-dhcp-checksum、uci-defaults/odhcpd.defaults、board.d/01_leds、board.d/02_network、board.d/99-default_network、config/dhcp、config/firewall、rc.d/K10gpio_switch、rc.d/K85odhcpd、rc.d/K90network、rc.d/S19dnsmasq、rc.d/S19firewall、rc.d/S20network、rc.d/S35odhcpd、rc.d/S94gpio_switch、modules.d/、modules-boot.d/、ppp/、sysctl.d/ 等，创建squashfs镜像，
x@x-Surface-Pro-3:~/下载$ mksquashfs openwrt-19.07.1-bcm53xx-generic-lxc openwrt-19.07.1-bcm53xx-lxc.bin
Parallel mksquashfs: Using 4 processors
Creating 4.0 filesystem on openwrt-19.07.1-bcm53xx-lxc.bin, block size 131072.
[=======================================================================================================/] 881/881 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
compressed data, compressed metadata, compressed fragments,
compressed xattrs, compressed ids
duplicates are removed
Filesystem size 3163.90 Kbytes (3.09 Mbytes)
48.94% of uncompressed filesystem size (6464.54 Kbytes)
Inode table size 9693 bytes (9.47 Kbytes)
25.72% of uncompressed inode table size (37693 bytes)
Directory table size 11372 bytes (11.11 Kbytes)
47.17% of uncompressed directory table size (24108 bytes)
Number of duplicate files found 136
Number of inodes 1145
Number of files 873
Number of fragments 39
Number of symbolic links171
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 101
Number of ids (unique uids + gids) 1
Number of uids 1
x (1000)
Number of gids 1
x (1000)
x@x-Surface-Pro-3:~/下载$
5、生成的openwrt-19.07.1-bcm53xx-lxc.bin传入光猫，'mtd' 写入mtd7分区。
刷之前先关掉saf，因saf有lxc-monitord进程守护，kill掉之后会自动运行，只好修改saf程序名，重启光猫。
root@F650A:~ # mv /usr/sbin/saf /usr/sbin/saf.bak
root@F650A:~ # reboot重启后telnet登录，'ps' 确认进程列表没有saf，备份然后写入并挂载。root@F650A:~ # ps
PID USER VSZ RSSSTATCPU COMMAND
1 root 23160 600S <1>init
......
root@F650A:~ # dd if=/dev/mtd7 of=/mnt/USB_disc1/mtd7.bin bs=1M
8+0 records in
8+0 records out
root@F650A:~ # mtd write /mnt/USB_disc1/openwrt-19.07.1-bcm53xx-lxc.bin /dev/mtd7
Unlocking /dev/mtd7 ...
Writing from /mnt/USB_disc1/openwrt-19.07.1-bcm53xx-lxc.bin to /dev/mtd7 ...
root@F650A:~ # mount /dev/mtdblock7 /opt/upt/framework1
6、开启OpenWRT新世界
写入mtd7分区并挂载之后，开始首次启动的准备工作。F650A的lxc默认路径在/usr/var/lib/lxc（一楼有说明）,按config文件创建相应文件及文件夹，
root@F650A:~ # mkdir -p /usr/var/lib/lxc/openwrt
root@F650A:~ # cd /usr/var/lib/lxc/openwrt
root@F650A:/usr/var/lib/lxc/openwrt # ln -s /opt/upt/framework1/pre-start.sh pre-start.sh
root@F650A:/usr/var/lib/lxc/openwrt # ln -s /opt/upt/framework1/config config
root@F650A:/usr/var/lib/lxc/openwrt # mkdir rootfs
root@F650A:/usr/var/lib/lxc/openwrt # mkdir /opt/upt/apps/openwrt-rootfs
root@F650A:/usr/var/lib/lxc/openwrt # lxc-start -dn openwrt
root@F650A:/usr/var/lib/lxc/openwrt # ps此时的进程列表应该可以看到/usr/sbin/uhttpd、dropbear等，或者可以尝试打开浏览器 http://光猫ip/或ssh登录，如果看到Authorization Required提示登录或ssh提示登录用户名密码，表明lxc已成功运行，但此时无密码状态无法网页和ssh登录，只能通过控制台方式添加密码后才能实现。关闭openwrt容器，'ps' 确认，前台启动。
root@F650A:/usr/var/lib/lxc/openwrt # lxc-stop -kn openwrt
root@F650A:/usr/var/lib/lxc/openwrt # ps
root@F650A:/usr/var/lib/lxc/openwrt # lxc-start -n openwrt
<6>init: Console is alive
<6>kmodloader: loading kernel modules from /etc/modules-boot.d/*
<6>kmodloader: done loading kernel modules from /etc/modules-boot.d/*
<6>init: - preinit -
WARNING: Variable 'network' does not exist or is not an array/object
WARNING: Variable 'lan' does not exist or is not an array/object
Press the key and hit to enter failsafe mode
Press the , , or key and hit to select the debug level
<5>mount_root: mounting /dev/root
urandom-seed: Seeding with /etc/urandom.seed
<6>procd: - early -
<6>procd: - ubus -
<6>procd: - init -
Please press Enter to activate this console.
<6>urngd: v1.0.2 started.此时回车便可看到openwrt的登入界面，添加密码。
BusyBox v1.30.1 () built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.||||.----.||_
| - ||_|-__| |||||| _|| _|
|_______|| __|_____|__|__||________||__||____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt 19.07.1, r10911-c155900f66
-----------------------------------------------------
root@openwrt:/# passwd root
Changing password for root
New password:
Bad password: too weak
Retype password:
passwd: password for root changed by root
root@openwrt:/# exit
Please press Enter to activate this console.控制台方式进入openwrt之后，exit无效，没找到正常的退出方式，只能关闭窗口，此后就可以通过网页和ssh正常登录。
7、自启动及进程守护
telnet登录光猫主系统,确认saf程序已备份，
root@F650A:~ # ls /usr/sbin/
chroot lsap saf.bak telnetd
root@F650A:~ # vi /usr/sbin/saf填入 (2021年5月更新：去除USB端口挂载判定，避免无挂载时LXC不能启动)
#!/bin/sh
# add user-defined startup scripts here
df | grep mtdblock7
if [ $? -ne 0 ]
then
/bin/mount -n -t squashfs -o ro /dev/mtdblock7 /opt/upt/framework1
fi
##
#ps | grep ntfsmount | grep -v grep
#if [ $? -eq 0 ]
# then
/dbus/usr/bin/lxc-start -dn openwrt
#fi
##
或者在电脑编辑好内容，命名为saf，传入/usr/sbin/，'chmod +x' 添加执行权限，重启光猫。
8、后续问题
目前的openwrt安装或更新程序会报错，缺少 libustream-openssl 、ca-bundle 、ca-certificates 和 libopenssl1.1，还需要修改dns，添加中文等
root@openwrt:~# opkg update
Downloading https://mirrors.cloud.tencent.com/lede/releases/19.07.1/targets/bcm53xx/generic/packages/Packages.gz
wget: SSL support not available, please install one of the libustream-.* packages as well as the ca-bundle and ca-certificates packages.
*** Failed to download the package list from https://mirrors.cloud.tencent.com/lede/releases/19.07.1/targets/bcm53xx/generic/packages/Packages.gz这些安装包可以从腾讯的镜像站下载然后手动安装。
root@openwrt:~# cd /mnt/USB_disc1/
root@openwrt:/mnt/USB_disc1# opkg install *.ipk修改dns
root@openwrt:/mnt/USB_disc1# rm /etc/resolv.conf
root@openwrt:/mnt/USB_disc1# vi /etc/resolv.conf填入，保存
nameserver 223.5.5.5
nameserver 8.8.8.8
nameserver 2001:da8::666
嫌过程繁琐的可以直接下载我配置好的镜像，从第5部分最后开始操作,将下载的openwrt-19.07.1-bcm53xx-lxc.bin和saf-lxc-start-openwrt放入光猫上的u盘，子系统默认root密码123456。
root@F650A:~ # dd if=/dev/mtd7 of=/mnt/USB_disc1/mtd7.bin bs=1M
root@F650A:~ # mtd write /mnt/USB_disc1/openwrt-19.07.1-bcm53xx-lxc.bin /dev/mtd7
root@F650A:~ # mount /dev/mtdblock7 /opt/upt/framework1
root@F650A:~ # mkdir -p /usr/var/lib/lxc/openwrt
root@F650A:~ # cd /usr/var/lib/lxc/openwrt
root@F650A:/usr/var/lib/lxc/openwrt # ln -s /opt/upt/framework1/pre-start.sh pre-start.sh
root@F650A:/usr/var/lib/lxc/openwrt # ln -s /opt/upt/framework1/config config
root@F650A:/usr/var/lib/lxc/openwrt # mkdir rootfs
root@F650A:/usr/var/lib/lxc/openwrt # mkdir /opt/upt/apps/openwrt-rootfs
root@F650A:/usr/var/lib/lxc/openwrt # lxc-start -dn openwrt
root@F650A:/usr/var/lib/lxc/openwrt # cd ~
root@F650A:~ # mv /usr/sbin/saf /usr/sbin/saf.bak
root@F650A:~ # cp /mnt/USB_disc1/saf-lxc-start-openwrt /usr/sbin/saf
root@F650A:~ # chmod 0755 /usr/sbin/saf 虽然子系统的openwrt可以顺利运行，但经过我的测试，kmod类app大多无法正常工作。比如F650A内核不支持ext4文件系统，在子系统安装kmod-fs-ext4,尽管可以安装成功，但重启后提示 modprobe 启用ext4模块失败。
子系统的uhttpd默认占用80和443端口、dropbear占用22端口，如果主系统有应用使用这些端口，会产生冲突，需修改rootfs/etc/config/下uhttpd和dropbear的端口设置。F650A的电信web管理系统尽管使用了8080端口，但web登录界面二者公用，当输入超密自动跳转至 :8080/start.ghtml，而使用光猫背面密码则跳转至80端口的luci，所以新的openwrt刷入后电信web管理系统将不可用，这意味着不能方便配置光猫网络，恢复出厂后无法重新拨号。为使子系统与主系统的web页面共存，即访问 http://光猫ip/ 时进入子系统luci页面，而访问 http://光猫ip:8080/ 时进入电信web管理页面，需要更改光猫主系统，此操作的定制空间更大，有机会的话在后面回复中整理出自定义光猫固件的思路。
期待大神的教程 谢谢分享，期待大神的教程 滿滿的乾貨，還具有很好的分享精神，贊一個！！ 坐等lxc的教程，另外19openwrt是你自己编译的还是直接下载armv7的官方bin刷进去的？
中兴非15.0.1出问题是你没清理/opt/upt/apps//apps目录
几乎所有光猫为了节省空间？都会有部分lib和var目录重用主系统的文件，openwrt下e8clib和e8cvar可以看看。
另外中兴编译的openwrt开始了uclibc的xlocale配置，这也是achaoge直接用omap源的APP无法运行的原因，我自己有手动开启xlocale后编译omap的openwrt放进去运行OK。
有兴趣可以一起研究玩。 anysoft 发表于 2020-2-20 12:58
坐等lxc的教程，另外19openwrt是你自己编译的还是直接下载armv7的官方bin刷进去的？
中兴非15.0.1出问题是 ...
openwrt1907用的官方rootfs经https://github.com/mikma/lxd-openwrt修改而来；xlocale的问题我在编译主系统的busybox时也发现了，官方用的应该是buildroot2015.08.1，如果有dtb和驱动甚至可以自己写个固件了。 感谢分享 songee 发表于 2020-2-20 21:40
openwrt1907用的官方rootfs经https://github.com/mikma/lxd-openwrt修改而来；xlocale的问题我在编译主 ...
是的， 中兴使用的是buildroot-2015.8.1，这个strings 一下sendcmd 就可以看到
看了下操作步骤，其实如果内存足够，没必要关闭saf启动的天翼网关。
可以让lxc容器启动两个op应该是可以的，F650A和华为hs8145V应该都是512MB内存，内存足够，openwrt可以放到apps分区文件夹下，不一定要写入mtd7再手动挂载。
我的HS8145V启动后内存只占用了30%，足够了。
另外主系统下的usr/sbin目录下文件不是内存加载的rootfs对应的squash文件？这个更改应该不会回写到文件系统，重启应该就会消失？
不晓得中兴和华为是不是一样的。可能中兴可以修改文件吧，华为的只能dd备份rootfs分区后自己解包修改打包后再dd写进去，会有砖的风险，砖了只能上编程器。
至于端口问题可以在打包openwrt的时候先修改每个服务对应监听端口就不会冲突了。