* [2024-06-28 Fri] ** well, i'm setting up sbcl and sly? because i imagine using Common Lisp for my attempt at building screen sharing for the XR glasses for the cool runtime access, right? ** well, just flake shell and (use-package sly) is enough for .lisp files to get stuff yes, i'm still using C-M-f and stuff to move around sexps but things are fine so, what's next? well, this is nice high level description dbus https://www.freedesktop.org/wiki/Software/dbus/ and here's something to start usage https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html from https://github.com/death/dbus and even lots of examples, but i think i will need a big book on Common Lisp to learn new things? * [2024-06-29 Sat] ** going through the dbus guide, sounds neat https://develop.kde.org/docs/features/d-bus/introduction_to_dbus/ that qdbus and qdbus-viewer are in #+begin_src bash $ nix shell nixpkgs#kdePackages.qttools #+end_src ** now let's read about calling methods * the receiving of the signals * all the different tabs i had ** overall freedesktop portal dbus api docs https://docs.flatpak.org/en/latest/portal-api-reference.html#gdbus-org.freedesktop.portal.ScreenCast ** searching github for 'get-managed-objects usage https://github.com/search?q=get-managed-objects&type=code ** searchin github for 'define-singlan-handler usage examples https://github.com/search?q=dbus%3Adefine-dbus-signal-handler&type=code ** example in cl-notify of calling a method 'with-introspected-object https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/dbus-protocol.lisp#L27 #+begin_src lisp (defmacro with-dbus-method-inovaction ((result-var method &rest args) &body body) (alexandria:with-gensyms (bus notifications-object) `(dbus:with-open-bus (,bus (dbus:session-server-addresses)) (dbus:with-introspected-object (,notifications-object ,bus "/org/freedesktop/Notifications" "org.freedesktop.Notifications") (let ((,result-var (,notifications-object "org.freedesktop.Notifications" ,method ,@args))) ,@body))))) #+end_src ** example in cl-notify of working with signals https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/signals.lisp they register single handler, and add callbacks dynamically & call them from the registered function ** dbus library example for publishing objects https://github.com/death/dbus/blob/master/examples/publish.lisp ** my request for help with interactive async calls https://github.com/death/dbus/issues/31 ** old ticket on dbus repo about enabling introspection https://github.com/death/dbus/issues/23 ** example of signal-handler in stumpwm-thingy https://github.com/lokedhs/stumpwm-dbus/blob/9cf0b52876111e777da27a42e3c81269b97c0005/src/pidgin.lisp#L16 ** ref for symbold in dbus package https://quickref.common-lisp.net/dbus.html#The-dbus_002fconnections-system ** blog article with basics of usage https://blog.macrolet.net/posts/DBus-and-PolicyKit-from-Common-Lisp.html ** big article about iolib https://pages.cs.wisc.edu/~psilord/blog/data/iolib-tutorial/tutorial.html it's the one that's used to create multiplexer for communication i tried to figure out what it means to have event-loops ** docs on dbus Screenshots portal https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Screenshot.html the methods and stuff ** docs on dbus type system https://dbus.freedesktop.org/doc/dbus-specification.html#type-system i.e how to understand types of methods and properties #+begin_src lisp CL-USER> (dbus:sigexp "xsaysas") (:INT64 :STRING (:ARRAY :BYTE) :STRING (:ARRAY :STRING)) #+end_src function signature explanation can show what should lisp types be https://github.com/death/dbus/issues/17#issuecomment-418920440 ** example of flameshot how they get Screenshot via dbus https://github.com/flameshot-org/flameshot/blob/c1dac52231024174faa68a29577129ebca125dff/src/utils/screengrabber.cpp#L59 ** dbus methods for Notification https://specifications.freedesktop.org/notification-spec/notification-spec-latest.html#signals ** dbus methods of ScreenCast https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html how to start them etc ** dbus spec on Request & Response for the async methods https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html#org-freedesktop-portal-request ** repo for xdg-portal for wayland https://github.com/emersion/xdg-desktop-portal-wlr ** the python snippet to check that ScreenCast via xdg-portal works https://gitlab.gnome.org/-/snippets/19 * [2024-07-20 Sat] ** and now i want to try to get display of stuff into gtk window, i guess ** doc about cl bindings to gtk3 https://www.crategus.com/books/cl-gtk/gtk-tutorial.html#example-window-simple-c ** well, i guess i could go and try to read guide on pipewire with C examples https://docs.pipewire.org/page_tutorial.html this is tutorial for the system and here is example program that maybe does similar thing https://docs.pipewire.org/video-play_8c-example.html * [2024-07-28 Sun] ** ok, now i'm trying to get frames from the pipewire 'out' node and one option i'm exploring is gstreamer which is a framework for creating applications that stream video / audio with some kind of ability to define \ declare pipeline and then start and then use api somehow there's c api, and gobject accessibility layer so either c to common lisp, or gobject common lisp thingy maybe exist ** reading docs https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/using.html?gi-language=c added via gst_all_1.gstreamer ** more readin gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink this is test for how it's installed needed to add plugins https://nixos.wiki/wiki/GStreamer ** i guess i can try to go with the tutorials? https://gstreamer.freedesktop.org/documentation/tutorials/index.html?gi-language=c but with common lisp over gobject? i think i'm already using this in the gtk bindings playground? maybe not https://cl-gtk2.common-lisp.dev/doc/gobject/Creating-GObjects-classes-and-implementing-GInterfaces.html #+begin_quote GObject binding at the moment provides only limited scenarios of creating GObject classes. #+end_quote ** and there's something https://github.com/BohongHuang/cl-gobject-introspection-wrapper which is based on older https://github.com/andy128k/cl-gobject-introspection https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=gobject they are packaged in nix, this is just unbelievable how much is done by humanity, omg * [2024-08-15 Thu] ** getting back after a long while. so, i was at the creating a window, not knowing how to stream video from pipewire out node so. what to do, huh i think gtk3 bindings library was separate from gobject and glib but maybe it doesn't matter? or maybe i should figure out glib bindings and use gtk3 though it? ** well, there's something simple with opening a window [[file:gtk-playground.lisp::defun example-window-simple (]] ** and something extra simple with gstreamer opening separate window for test video source [[file:gstreamer/tutorial-hello-world.lisp]] ** maybe try to open gstreamer window with screen share? ok, when i try example of pipewire window on screenshare source i get error that pipewiresrc is not found and that's right, i used system-wide gst-launch-1.0 and when i tried with one in flake shell - it also didn't work so. tried removing all from shell env, then not found namespace Gst (defvar *gstreamer* (gir:require-namespace "Gst")) so. yeah. maybe i need to install dev headers or something? ** so. for moving the video around. chatgpt recommended something named 'videobox' like i can declare it in the pipeline and then get reference to it and use to set video coordinates, huh? https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=python the thing exists but i have zero idea whether it's accessible in a way that's recommended #+begin_src lisp (defun for-screencast () (gir:invoke (*gstreamer* 'init) '()) (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch) "pipewiresrc path=49 ! videoconvert ! videobox top=0 left=0 right=0 bottom=0 ! videoconvert ! autovideosink" )) (videobox (gir:invoke (pipeline 'get-element-by-name) "videobox"))) ;; Set the initial state to playing (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing)) ;; Function to move the video (defun move-video (top left) (gir:invoke (videobox 'set-property) 'top top) (gir:invoke (videobox 'set-property) 'left left)) ;; Example: Move the video 10 pixels down and 20 pixels to the right (move-video 10 20))) #+end_src and similar advice from claude, huh #+begin_src lisp (defun for-screencast () (gir:invoke (*gstreamer* 'init) '()) (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch) "pipewiresrc path=49 ! videoconvert ! videobox name=move ! videoscale ! autovideosink")) (move-element (gir:invoke (pipeline 'get-by-name) "move"))) (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing)) (values pipeline move-element))) (defun move-video (move-element left top) (gir:set-property move-element "left" left) (gir:set-property move-element "top" top)) ;; Usage: (multiple-value-bind (pipeline move-element) (for-screencast) ;; Move video 50 pixels to the right and 30 pixels down (move-video move-element 50 30) ;; Your main loop or other logic here ) #+end_src let's try this out * [2024-08-16 Fri] ** new day of the vacation the experiment with moving the video, right? could it be that both examples are fake? i don't seem to find both 'get-element things trying to find anything about dynamism in gstreamer https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/ looks like this is about changing elements of the pipeline and it's possible that parameters of the pipeline elements (pads?) are also static ** options for transforming frames? *** videotransform : A combination of scaling, 90-degree-step rotation, https://github.com/Freescale/gstreamer-imx/blob/master/README.md horizontal/vertical flipping, and color space conversion operations. These elements are able to render video overlay compositions from GstVideoOverlayCompositionMeta data. They also implement the GstVideoDirection interface and have a "video-direction" property that handles rotation and flipping. Through this property, it is possible to configure these elements to auto-rotate images according to the information in image-orientation tags. *** glimagesink https://gstreamer.freedesktop.org/documentation/opengl/glimagesink.html?gi-language=python gst-launch-1.0 -v videotestsrc ! video/x-raw ! glimagesink has error ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to create EGLDisplay from native display Additional debug info: ../ext/gl/gstglimagesink.c(1136): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink ERROR: pipeline doesn't want to preroll. so maybe opengl is not available? how to check? *** gltransformation https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=python here examples seem to be for images, not video and same error, let's figure out the opengl check ** somewhat categorized things? * [2024-08-21 Wed] ** visiting sister, returning back last time i got stuck on video displaying frozen when i tried to add appsink just re-checking my basic gstreamer pipeline and screencast start - thing works but i want to clean it up i guess, to allow for simpler retries? ** so, for the appsnk there are signals https://gstreamer.freedesktop.org/documentation/app/appsink.html?gi-language=python#appsink::new-sample new-sample #+begin_src python def new_sample_callback (appsink, udata): #python callback for the 'new-sample' signal #+end_src Signal that a new sample is available. but how do i connect to a callback from common lisp and gobject introspection https://github.com/andy128k/cl-gobject-introspection does the gitlab repo has info on callbacks? can't seem to call method 'pull-sample' maybe article will help? https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=python#grabbing-data-with-appsink well, this looks like method on gst namespace? "gst_app_sink_pull_sample()" https://gstreamer.freedesktop.org/documentation/applib/gstappsink.html?gi-language=python#gst_app_sink_pull_sample there's article for python bindings https://lifestyletransfer.com/how-to-use-gstreamer-appsink-in-python/ * [2024-08-22 Thu] ** hey, i think this is progress i am able to call appsink 'pull-sample after all (for some reason, i don't know what i was doing differently) new error: >>> got to process sample with # but in the code for processing of the sample some type errors Backtrace: 0: ((:METHOD GIR::THIS-OF (T)) T) [fast-method] 1: (GIR:PROPERTY T DATA) 2: (PROCESS-FRAME #) 3: ((LAMBDA (APPSINK) :IN START-PROCESSING) #) ok, but when i have struct instance, how do i introspect it? ** let's just go and read documentation for the type that's returned from pull-sample, sure also there's that python code, right? doc 1: get sample on appsink https://people.freedesktop.org/~tsaunier/documentation/gst-plugins-base-app-1.0/gstappsink.html?gi-language=c#gst_app_sink_pull_sample doc 2: GstSample class https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=c#GstSample so on GstSample everything is taken as results of funciton calls? let's try that well, maybe i don't need get-info on sample, it returns null, maybe i need it on buffer? doc 3: get-buffer on sample https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=python#gst_sample_get_buffer doc 4: buffer class info https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_peek_memory ** i'm recommended to look into GstBuffer.map https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map it returns some https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstmemory.html?gi-language=python#GstMapInfo ** struggling, searching for answers https://github.com/andy128k/cl-gobject-introspection/issues/73#issuecomment-755316735 Examples of inspection of a repository can be found in test files. https://github.com/andy128k/cl-gobject-introspection/blob/master/test/test1.lisp and i mean it's possible that referencing members of a structure is not implemented? so, huh using internal functions #+begin_src lisp (data (gir:fields-dict-of (gir:struct-class-of map-info))) #+end_src i see >>>> and data has size NIL and buf ((memory . #) (flags . #) (data . #) (size . #) (maxsize . #) (user_data . #) (_gst_reserved . #)) ; Evaluation aborted on T CAPTURING-FRAMES> so. there are data and size fields. maybe i can try with ** well, for some reason data is not avaiable? (let* ((buffer (gir:invoke (sample 'get-buffer))) (info (gir:invoke (sample 'get-info))) (caps (gir:invoke (sample 'get-caps))) (caps-struct (gir:invoke (caps 'get-structure) 0)) (width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int))) (height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int))) (buf-size (gir:invoke (buffer 'get-size))) (data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size))) (format t ">>>> at buffer got size ~A and tried to copy ~A~%" buf-size data-copy) ;; ok, we have buffer, let's try to get copy ;; (multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget *gstreamer* "MapFlags" :read)) ;; (format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset)) ;; (format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string))) ;; (format t ">>>> the fields are width ~A, height ~A~%" width height) ;; (format t ">>>> and map result are ~A and ~A~%" found map-info) ;; (when found ;; (let ((data (gir:field map-info 'memory)) ;; (size (gir:field map-info 'size))) ;; (format t ">>>> and struct info? ~A~%" map-info) ;; (gir:invoke (buffer 'unmap) map-info) ;; (format t ">>>> and data has size ~A and memory is ~A~%" size (or data "empty data"))))) ) data-copy is NIL yep. mda. ** well, trying to copy as in python example didn't work either (buf-size (gir:invoke (buffer 'get-size))) (data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size)) https://github.com/jackersson/gst-python-tutorials/blob/master/launch_pipeline/run_appsink.py#L49 yeah, it's possible that gir library doesn't work well? even if on c level all is ok? * [2024-08-23 Fri] ** well, problem is in trying to read gir:map-c-array it gets length nil and fails GIR::MAP-C-ARRAY so what i'm using: https://github.com/andy128k/cl-gobject-introspection and there's wrapper around it https://github.com/bohonghuang/cl-gobject-introspection-wrapper but there is also https://github.com/kat-co/gir2cl maybe it will work? or maybe i'll figure out what's up ** well, trying to check available slots on that array type (mapcar #'c2mop:slot-definition-name (c2mop:class-slots (class-of c-array-type))) => (PARAM-TYPE FIXED-SIZE ZERO-TERMINATED? LENGTH) and how do i get their values? if i don't know name of the :reader ? ( from article on https://riptutorial.com/common-lisp/example/9826/obtain-the-slot-names-of-a-class ) but yeah. huh. ** let's write up the problem? I'm trying to use gstreamer - a framework for video\audio pipelines with gobject api the hope is to use their 'appsink' to get video frame data into the program i'm writing https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map i'll link the code in case it will be needed for replication, right now it seems that by doing gir calls i've successfully set up pipeline and receiving callbacks with the buffers the problem i get is when i try to access 'data and it seems to come from GIR:MAP-C-ARRAY where length of the parameter c-array-type is NIL in #'map-c-array it comes by calling #+begin_src lisp (defun c-array-length (c-array-type) (with-slots (param-type fixed-size zero-terminated? length) c-array-type (if fixed-size (if (and length (< length fixed-size)) length fixed-size) (if length (+ length (if zero-terminated? 1 0)))))) #+end_src and manually checking slots of the 'c-array-type it has NIL in length right now I don't know whether it's a problem on the side of cl-gobject-introspection in how it's initializing the c-array-type or maybe it's somehow problem on the side of the gstreamer library, that provides bad responses, i'll try to use other languages \ libraries but could also really use some help in figuring this out fuller log: #+begin_src The value NIL is not of type REAL [Condition of type TYPE-ERROR] Restarts: 0: [ABORT] abort thread (#) Backtrace: 0: (SB-KERNEL:TWO-ARG->= 0 NIL) 1: (GIR::MAP-C-ARRAY # #.(SB-SYS:INT-SAP #X7F1EC80EBBD8) #) Locals: ARRAY = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8) C-ARRAY-TYPE = # FUNC = # I = 0 LENGTH = NIL PARAM-SIZE = 1 POS = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8) 2: ((:METHOD GIR::MEM-GET (T GIR::POINTER-TYPE)) #.(SB-SYS:INT-SAP #X7F1F97F7FFF8) #) [fast-method] Locals: GIR::POS = #.(SB-SYS:INT-SAP #X7F1F97F7FFF8) TYPE = # 3: (GIR.FIELD:GET #.(SB-SYS:INT-SAP #X7F1ECC0036F0) #) 4: (PROCESS-FRAME #) 5: ((LAMBDA (APPSINK) :IN START-PROCESSING) #) 6: ((LAMBDA (GIR::CLOSURE RETURN GIR::N-VALUES GIR::PARAMS GIR::HINT GIR::DATA) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") #.(SB-SYS:INT-SAP #X7F1FCC470B30) #.(SB-S.. 7: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") # # #<.. 8: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN)) 9: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN)) 10: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN)) 11: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN)) 12: (SB-THREAD::RUN) 13: (SB-THREAD::ENTER-FOREIGN-CALLBACK 87 69885224809132 69885224809124) 14: ("foreign function: call_into_lisp_") 15: ("foreign function: funcall3") 16: ("foreign function: callback_wrapper_trampoline") 17: ("foreign function: #x501053AB") --more-- #+end_src the source: ( in the git : https://git.sunshine.industries/efim/scratch-screen-share/src/commit/73316581a056a20e51dc36eb96673acfee7d90b0/gstreamer/capturing-frames.lisp ) #+begin_src lisp ;(load (sb-ext:posix-getenv "ASDF")) (asdf:load-system 'cl-cffi-gtk) (asdf:load-system 'cl-gobject-introspection) (defpackage :capturing-frames (:use :cl)) (in-package :capturing-frames) (defvar *gstreamer* (gir:require-namespace "Gst")) ;(defvar *gstapp* (gir:require-namespace "GstApp")) (defun cast-with-videobox (&optional stream-num) (gir:invoke (*gstreamer* 'init) '()) (let* ((pipeline (gir:invoke (*gstreamer* "Pipeline" 'new) "video-pipeline")) (source (gir:invoke (*gstreamer* "ElementFactory" 'make) (if stream-num "pipewiresrc" "videotestsrc") "source")) (convert (gir:invoke (*gstreamer* "ElementFactory" 'make) "videoconvert" "convert")) (tee (gir:invoke (*gstreamer* "ElementFactory" 'make) "tee" "tee")) (queue1 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue1")) (queue2 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue2")) (sink (gir:invoke (*gstreamer* "ElementFactory" 'make) "autovideosink" "sink")) (appsink (gir:invoke (*gstreamer* "ElementFactory" 'make) "appsink" "appsink")) ) (when stream-num ;; Set pipewiresrc properties, such as the path (setf (gir:property source 'path) (format nil "~A" stream-num))) ;; Add and link elements in the pipeline (gir:invoke (pipeline 'add) source) (gir:invoke (pipeline 'add) convert) (gir:invoke (pipeline 'add) tee) (gir:invoke (pipeline 'add) queue1) (gir:invoke (pipeline 'add) queue2) (gir:invoke (pipeline 'add) sink) (gir:invoke (pipeline 'add) appsink) (gir:invoke (source 'link) convert) (gir:invoke (convert 'link) tee) (gir:invoke (tee 'link) queue1) (gir:invoke (tee 'link) queue2) (gir:invoke (queue1 'link) sink) (gir:invoke (queue2 'link) appsink) ;; Configure appsink (setf (gir:property appsink "emit-signals") t (gir:property appsink "max-buffers") 1 (gir:property appsink "drop") t) (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing)) (values pipeline appsink))) (defun process-frame (sample) (format t ">>> got to process sample with ~A~%" sample) (let* ((buffer (gir:invoke (sample 'get-buffer))) (info (gir:invoke (sample 'get-info))) (caps (gir:invoke (sample 'get-caps))) (caps-struct (gir:invoke (caps 'get-structure) 0)) (width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int))) (height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int)))) (multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget *gstreamer* "MapFlags" :read)) (format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset)) (format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string))) (format t ">>>> the fields are width ~A, height ~A~%" width height) (format t ">>>> and map result are ~A and ~A~%" found map-info) (when found (let ((data (gir:field map-info 'data)) (size (gir:field map-info 'size))) (format t ">>>> and struct info? ~A~%" map-info) (gir:invoke (buffer 'unmap) map-info) (format t ">>>> and data has size ~A and data is ~A~%" size (or data "empty data"))))))) (defun start-processing (appsink) (gir:connect appsink "new-sample" (lambda (appsink) (format t ">> got printed object ~A~%" appsink) (format t ">> got printed class ~A~%" (gir:gir-class-of appsink)) (format t ">> trying to get method on sink ~A~%" (gir:nget appsink 'pull-sample)) (let ((sample (gir:invoke (appsink 'pull-sample)))) (when sample (process-frame sample))) (gir:nget *gstreamer* "FlowReturn" :ok)))) (defun try-run () (multiple-value-bind (pipeline appsink) (cast-with-videobox) (format t "got appsink ~A" appsink) (start-processing appsink) (loop (sleep 1)))) ;(try-run) #+end_src *** submitted https://github.com/andy128k/cl-gobject-introspection/issues/102 ** what's next? i could try another library for gir bindings. not sure whether that would help much, but yeah or i could just try to replicate this in python or maybe in go? ** whelp gir2cl is also on top of cl-gobject-introspection huh. let's set up a directory with go stuff? i guess trying in separate directory with https://github.com/linuxdeepin/go-gir but more realistically https://pkg.go.dev/github.com/go-gst/go-gst this is very much different, but yeah. let's go with appsink example https://pkg.go.dev/github.com/go-gst/go-gst@v1.1.0/examples/appsink for the go library to pick up gstreamer from nix shell https://discourse.nixos.org/t/flakes-how-to-make-pkg-config-work-with-nix-shell/16305 it has to somehow be exposed via pkg-config yes, adding pkg-config to shell works, nice * [2024-08-24 Sat] ** ok, continuing with go thing. into what do i want to decode stuff? well, the reported default format is sample info: video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, format=(string)YV12, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive http://www.avisynth.nl/index.php/FAQ_YV12 but maybe i want something else? let's read docs on appsrc https://github.com/go-gst/go-gst/tree/v1.1.0/examples/appsrc * [2024-12-08 Sun] ** well, i guess i will maybe return to this? https://www.youtube.com/watch?v=Jn_09BeUEI0 here is gltransformation this looks like what i'd really want, no? ** and this is documentation https://gstreamer.freedesktop.org/documentation/opengl/index.html?gi-language=c https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=c#gltransformation why doesn't it work? * [2024-12-21 Sat] ** wait. it works now? how and why maybe because it's installed via apt? yes. if i try to use gst-launch-1.0 gltestsrc ! gltransformation scale-y=0.5 scale-x=0.5 ! glimagesink in the nix shell i get #+begin_src Setting pipeline to PAUSED ... Caught SIGSEGV Spinning. Please run 'gdb gst-launch-1.0 242683' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core. #+end_src ** for now found article about other ways to transform https://lubosz.wordpress.com/2014/06/16/transforming-video-on-the-gpu/ but that this opengl way is much faster comparing with this: https://gstreamer.freedesktop.org/documentation/videoconvertscale/videoscale.html?gi-language=c gst-launch-1.0 -v videotestsrc ! videoscale ! autovideosink gst-launch-1.0 -v videotestsrc ! videoscale ! video/x-raw,width=1000 ! autovideosink ** the ordinary error is gst-launch-1.0 gltestsrc ! gltransformation rotation-z=45 ! glimagesink Setting pipeline to PAUSED ... Caught SIGSEGV Spinning. Please run 'gdb gst-launch-1.0 7843' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core. ** huh. trying previous nixpkgs releases and in 23.05 the error is different efim-nefedov@LLF33A87M:~/Documents/personal/scratch-screen-share$ gst-launch-1.0 gltestsrc ! gltransformation rotation-z=45 ! glimagesink Setting pipeline to PAUSED ... ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Could not find any FBConfig's to use (check attributes?) Additional debug info: ../ext/gl/gstglimagesink.c(1131): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. Setting pipeline to NULL ... Freeing pipeline ... ** ok, looking at the error the problem is in creating connection WARN gldisplayegl gstgldisplay_egl.c:408:gst_gl_display_egl_from_gl_display: failed to get EGLDisplay from native display but beats me if i know what needs to be done claude.ai recommends adding LD_LIBRARY_PATH to gstreamer so trying #+begin_src nix gstWrapper = pkgs.writeScriptBin "gst-launch-wrapped" '' #!${pkgs.bash}/bin/bash export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH exec ${pkgs.gst_all_1.gstreamer}/bin/gst-launch-1.0 "$@" ''; #+end_src but that fails $ gst-launch-wrapped gltestsrc ! gltransformation rotation-z=45 ! glimagesink /nix/store/p6k7xp1lsfmbdd731mlglrdj2d66mr82-bash-5.2p37/bin/bash: error while loading shared libraries: __vdso_gettimeofday: invalid mode for dlopen(): Invalid argument * [2024-12-22 Sun] ** ok, let's try what let's try default video out after i turn on the screen sharing from cl part with code cleaned up i can start the share, that's nice gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! autovideosink let's also try to add opengl processing gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! gltransformation rotation-z=45 ! glimagesink so, this kind of works gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-z=45 ! glimagesink even if the thing is small ** trying to figure out why gl things open a small window *** one that works GST_DEBUG=4,caps:5 gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! autovideosink has deterministic way of creating a stream-id 0:00:00.032812186 44186 0x74b020000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw, format=(string)BGRA, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1 0:00:00.033493070 44186 0x74b020000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)YV12 *** and comparing with GST_DEBUG=4,caps:5 gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-z=45 ! glimagesink (by recommendation of claude) 0:00:00.152714238 44533 0x784a70000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)RGBA, texture-target=(string)2D 0:00:00.152789062 44533 0x784a70000b70 INFO basetransform gstbasetransform.c:1326:gst_base_transform_setcaps: reuse caps 0:00:00.152821256 44533 0x784a70000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)RGBA, texture-target=(string)2D well, i guess i can just open the window full size *** ok! looks like it does stream full resolution. now. i'd want to open the displaying pipeline programmatically and then maybe try to dynamically change params i guess even without a window embedding, just starting a window is good enough but also, hm. do i need any ui for the video window? i really don't think so. maybe let's figure out the missing cursor gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-x=-15 ortho=True ! glimagesink gst-launch-1.0 gltestsrc ! gltransformation rotation-x=-45 ortho=True ! glimagesink ** am i not getting into Create Session? after the Start? https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html (also note that here are cursor options) ** ok, let's try writing this in java then