scratch-screen-share/notes.org at master

  (defmacro with-dbus-method-inovaction ((result-var method &rest args) &body body)
  (alexandria:with-gensyms (bus notifications-object)
    `(dbus:with-open-bus (,bus (dbus:session-server-addresses))
       (dbus:with-introspected-object (,notifications-object ,bus "/org/freedesktop/Notifications" "org.freedesktop.Notifications")
         (let ((,result-var (,notifications-object "org.freedesktop.Notifications" ,method ,@args)))
          ,@body)))))

example in cl-notify of working with signals

https://github.com/Lautaro-Garcia/cl-notify/blob/main/src/signals.lisp

they register single handler, and add callbacks dynamically & call them from the registered function

it's the one that's used to create multiplexer for communication i tried to figure out what it means to have event-loops

docs on dbus Screenshots portal

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Screenshot.html

the methods and stuff

docs on dbus type system

https://dbus.freedesktop.org/doc/dbus-specification.html#type-system

i.e how to understand types of methods and properties

  CL-USER> (dbus:sigexp "xsaysas")
(:INT64 :STRING (:ARRAY :BYTE) :STRING (:ARRAY :STRING))

function signature explanation can show what should lisp types be https://github.com/death/dbus/issues/17#issuecomment-418920440

example of flameshot how they get Screenshot via dbus

https://github.com/flameshot-org/flameshot/blob/c1dac52231024174faa68a29577129ebca125dff/src/utils/screengrabber.cpp#L59

dbus methods for Notification

https://specifications.freedesktop.org/notification-spec/notification-spec-latest.html#signals

dbus methods of ScreenCast

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html how to start them etc

dbus spec on Request & Response for the async methods

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.Request.html#org-freedesktop-portal-request

repo for xdg-portal for wayland

https://github.com/emersion/xdg-desktop-portal-wlr

the python snippet to check that ScreenCast via xdg-portal works

https://gitlab.gnome.org/-/snippets/19

[2024-07-20 Sat]

and now i want to try to get display of stuff into gtk window, i guess

doc about cl bindings to gtk3

https://www.crategus.com/books/cl-gtk/gtk-tutorial.html#example-window-simple-c

well, i guess i could go and try to read guide on pipewire with C examples

https://docs.pipewire.org/page_tutorial.html this is tutorial for the system

and here is example program that maybe does similar thing https://docs.pipewire.org/video-play_8c-example.html

[2024-07-28 Sun]

ok, now i'm trying to get frames from the pipewire 'out' node

and one option i'm exploring is gstreamer

which is a framework for creating applications that stream video / audio with some kind of ability to define \ declare pipeline and then start and then use api somehow

there's c api, and gobject accessibility layer so either c to common lisp, or gobject common lisp thingy maybe exist

reading docs

https://gstreamer.freedesktop.org/documentation/frequently-asked-questions/using.html?gi-language=c

added via gst_all_1.gstreamer

i guess i can try to go with the tutorials?

https://gstreamer.freedesktop.org/documentation/tutorials/index.html?gi-language=c

but with common lisp over gobject? i think i'm already using this in the gtk bindings playground? maybe not https://cl-gtk2.common-lisp.dev/doc/gobject/Creating-GObjects-classes-and-implementing-GInterfaces.html

GObject binding at the moment provides only limited scenarios of creating GObject classes.

and there's something

https://github.com/BohongHuang/cl-gobject-introspection-wrapper

which is based on older

https://github.com/andy128k/cl-gobject-introspection

https://search.nixos.org/packages?channel=unstable&from=0&size=50&sort=relevance&type=packages&query=gobject

they are packaged in nix, this is just unbelievable how much is done by humanity, omg

[2024-08-15 Thu]

getting back after a long while.

so, i was at the creating a window, not knowing how to stream video from pipewire out node so. what to do, huh i think gtk3 bindings library was separate from gobject and glib

but maybe it doesn't matter? or maybe i should figure out glib bindings and use gtk3 though it?

well, there's something simple with opening a window

gtk-playground.lisp::defun example-window-simple (

and something extra simple with gstreamer opening separate window for test video source

/efim/scratch-screen-share/src/branch/master/gstreamer/tutorial-hello-world.lisp

maybe try to open gstreamer window with screen share?

ok, when i try example of pipewire window on screenshare source i get error that pipewiresrc is not found and that's right, i used system-wide gst-launch-1.0 and when i tried with one in flake shell - it also didn't work

so. tried removing all from shell env, then not found namespace Gst (defvar gstreamer (gir:require-namespace "Gst"))

so. yeah. maybe i need to install dev headers or something?

so. for moving the video around. chatgpt recommended something named 'videobox'

like i can declare it in the pipeline and then get reference to it and use to set video coordinates, huh? https://gstreamer.freedesktop.org/documentation/videobox/index.html?gi-language=python the thing exists but i have zero idea whether it's accessible in a way that's recommended

(defun for-screencast ()
  (gir:invoke (*gstreamer* 'init) '())
  (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch)
                                "pipewiresrc path=49 ! videoconvert ! videobox top=0 left=0 right=0 bottom=0 ! videoconvert ! autovideosink"
                                ))
         (videobox (gir:invoke (pipeline 'get-element-by-name) "videobox")))
    ;; Set the initial state to playing
    (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
    
    ;; Function to move the video
    (defun move-video (top left)
      (gir:invoke (videobox 'set-property) 'top top)
      (gir:invoke (videobox 'set-property) 'left left))

    ;; Example: Move the video 10 pixels down and 20 pixels to the right
    (move-video 10 20)))

and similar advice from claude, huh

(defun for-screencast ()
  (gir:invoke (*gstreamer* 'init) '())
  (let* ((pipeline (gir:invoke (*gstreamer* 'parse-launch)
                               "pipewiresrc path=49 ! videoconvert ! videobox name=move ! videoscale ! autovideosink"))
         (move-element (gir:invoke (pipeline 'get-by-name) "move")))
    (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
    (values pipeline move-element)))

(defun move-video (move-element left top)
  (gir:set-property move-element "left" left)
  (gir:set-property move-element "top" top))

;; Usage:
(multiple-value-bind (pipeline move-element) (for-screencast)
  ;; Move video 50 pixels to the right and 30 pixels down
  (move-video move-element 50 30)
  ;; Your main loop or other logic here
  )

let's try this out

[2024-08-16 Fri]

new day of the vacation

the experiment with moving the video, right?

could it be that both examples are fake? i don't seem to find both 'get-element things

trying to find anything about dynamism in gstreamer https://coaxion.net/blog/2014/01/gstreamer-dynamic-pipelines/ looks like this is about changing elements of the pipeline

and it's possible that parameters of the pipeline elements (pads?) are also static

options for transforming frames?

videotransform : A combination of scaling, 90-degree-step rotation,

https://github.com/Freescale/gstreamer-imx/blob/master/README.md horizontal/vertical flipping, and color space conversion operations. These elements are able to render video overlay compositions from GstVideoOverlayCompositionMeta data. They also implement the GstVideoDirection interface and have a "video-direction" property that handles rotation and flipping. Through this property, it is possible to configure these elements to auto-rotate images according to the information in image-orientation tags.

glimagesink

https://gstreamer.freedesktop.org/documentation/opengl/glimagesink.html?gi-language=python gst-launch-1.0 -v videotestsrc ! video/x-raw ! glimagesink has error ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Failed to create EGLDisplay from native display Additional debug info: ../ext/gl/gstglimagesink.c(1136): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink ERROR: pipeline doesn't want to preroll.

so maybe opengl is not available? how to check?

gltransformation

https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=python

here examples seem to be for images, not video and same error, let's figure out the opengl check

somewhat categorized things?

[2024-08-21 Wed]

visiting sister, returning back

last time i got stuck on video displaying frozen when i tried to add appsink

just re-checking my basic gstreamer pipeline and screencast start - thing works but i want to clean it up i guess, to allow for simpler retries?

so, for the appsnk there are signals

https://gstreamer.freedesktop.org/documentation/app/appsink.html?gi-language=python#appsink::new-sample

new-sample

def new_sample_callback (appsink, udata):
    #python callback for the 'new-sample' signal

Signal that a new sample is available.

but how do i connect to a callback from common lisp and gobject introspection https://github.com/andy128k/cl-gobject-introspection does the gitlab repo has info on callbacks?

can't seem to call method 'pull-sample' maybe article will help? https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html?gi-language=python#grabbing-data-with-appsink

well, this looks like method on gst namespace? "gst_app_sink_pull_sample()" https://gstreamer.freedesktop.org/documentation/applib/gstappsink.html?gi-language=python#gst_app_sink_pull_sample

there's article for python bindings https://lifestyletransfer.com/how-to-use-gstreamer-appsink-in-python/

[2024-08-22 Thu]

hey, i think this is progress

i am able to call appsink 'pull-sample after all (for some reason, i don't know what i was doing differently)

new error: >>> got to process sample with #<STRUCT-INSTANCE {10128D02F3}> but in the code for processing of the sample some type errors

Backtrace: 0: ((:METHOD GIR::THIS-OF (T)) T) [fast-method] 1: (GIR:PROPERTY T DATA) 2: (PROCESS-FRAME #<GIR::STRUCT-INSTANCE {100BCDAB73}>) 3: ((LAMBDA (APPSINK) :IN START-PROCESSING) #<GIR::OBJECT-INSTANCE {100BBE02D3}>)

ok, but when i have struct instance, how do i introspect it?

let's just go and read documentation for the type that's returned from pull-sample, sure

also there's that python code, right? doc 1: get sample on appsink https://people.freedesktop.org/~tsaunier/documentation/gst-plugins-base-app-1.0/gstappsink.html?gi-language=c#gst_app_sink_pull_sample doc 2: GstSample class https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=c#GstSample

so on GstSample everything is taken as results of funciton calls? let's try that

well, maybe i don't need get-info on sample, it returns null, maybe i need it on buffer? doc 3: get-buffer on sample https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstsample.html?gi-language=python#gst_sample_get_buffer doc 4: buffer class info https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_peek_memory

i'm recommended to look into GstBuffer.map

https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map

it returns some https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstmemory.html?gi-language=python#GstMapInfo

struggling, searching for answers

https://github.com/andy128k/cl-gobject-introspection/issues/73#issuecomment-755316735 Examples of inspection of a repository can be found in test files.

https://github.com/andy128k/cl-gobject-introspection/blob/master/test/test1.lisp

and i mean it's possible that referencing members of a structure is not implemented?

so, huh using internal functions

(data (gir:fields-dict-of (gir:struct-class-of map-info)))

i see >>>> and data has size NIL and buf ((memory . #<FIELD-INFO {100FEB4083}>) (flags . #<FIELD-INFO {100FEB4153}>) (data . #<FIELD-INFO {100FEB4223}>) (size . #<FIELD-INFO {100FEB42F3}>) (maxsize . #<FIELD-INFO {100FEB43C3}>) (user_data . #<FIELD-INFO {100FEB4493}>) (_gst_reserved . #<FIELD-INFO {100FEB4563}>)) ; Evaluation aborted on T CAPTURING-FRAMES>

so. there are data and size fields.

maybe i can try with

well, for some reason data is not avaiable?

(let* ((buffer (gir:invoke (sample 'get-buffer))) (info (gir:invoke (sample 'get-info))) (caps (gir:invoke (sample 'get-caps))) (caps-struct (gir:invoke (caps 'get-structure) 0)) (width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int))) (height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int))) (buf-size (gir:invoke (buffer 'get-size))) (data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size))) (format t ">>>> at buffer got size ~A and tried to copy ~A~%" buf-size data-copy) ;; ok, we have buffer, let's try to get copy

;; (multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget gstreamer "MapFlags" :read)) ;; (format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset)) ;; (format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string))) ;; (format t ">>>> the fields are width ~A, height ~A~%" width height) ;; (format t ">>>> and map result are ~A and ~A~%" found map-info) ;; (when found ;; (let ((data (gir:field map-info 'memory)) ;; (size (gir:field map-info 'size))) ;; (format t ">>>> and struct info? ~A~%" map-info) ;; (gir:invoke (buffer 'unmap) map-info) ;; (format t ">>>> and data has size ~A and memory is ~A~%" size (or data "empty data"))))) )

data-copy is NIL yep. mda.

well, trying to copy as in python example didn't work either

(buf-size (gir:invoke (buffer 'get-size))) (data-copy (gir:invoke (buffer 'extract-dup) 0 buf-size))

https://github.com/jackersson/gst-python-tutorials/blob/master/launch_pipeline/run_appsink.py#L49

yeah, it's possible that gir library doesn't work well? even if on c level all is ok?

[2024-08-23 Fri]

well, problem is in trying to read gir:map-c-array

it gets length nil and fails

GIR::MAP-C-ARRAY

so what i'm using: https://github.com/andy128k/cl-gobject-introspection and there's wrapper around it https://github.com/bohonghuang/cl-gobject-introspection-wrapper

but there is also https://github.com/kat-co/gir2cl maybe it will work? or maybe i'll figure out what's up

well, trying to check available slots on that array type

(mapcar #'c2mop:slot-definition-name (c2mop:class-slots (class-of c-array-type))) => (PARAM-TYPE FIXED-SIZE ZERO-TERMINATED? LENGTH)

and how do i get their values? if i don't know name of the :reader ? ( from article on https://riptutorial.com/common-lisp/example/9826/obtain-the-slot-names-of-a-class )

but yeah. huh.

let's write up the problem?

I'm trying to use gstreamer - a framework for video\audio pipelines with gobject api

the hope is to use their 'appsink' to get video frame data into the program i'm writing https://people.freedesktop.org/~tsaunier/documentation/gstreamer-1.0/gstbuffer.html?gi-language=python#gst_buffer_map

i'll link the code in case it will be needed for replication, right now it seems that by doing gir calls i've successfully set up pipeline and receiving callbacks with the buffers

the problem i get is when i try to access 'data and it seems to come from GIR:MAP-C-ARRAY where length of the parameter c-array-type is NIL

in #'map-c-array it comes by calling

(defun c-array-length (c-array-type)
  (with-slots (param-type fixed-size zero-terminated? length)
      c-array-type
    (if fixed-size
	(if (and length (< length fixed-size))
	    length
	    fixed-size)
	(if length
	    (+ length (if zero-terminated? 1 0))))))

and manually checking slots of the 'c-array-type it has NIL in length

right now I don't know whether it's a problem on the side of cl-gobject-introspection in how it's initializing the c-array-type or maybe it's somehow problem on the side of the gstreamer library, that provides bad responses, i'll try to use other languages \ libraries

but could also really use some help in figuring this out

fuller log:

The value
  NIL
is not of type
  REAL
   [Condition of type TYPE-ERROR]

Restarts:
 0: [ABORT] abort thread (#<FOREIGN-THREAD tid=127655 "callback" RUNNING {10131D8033}>)

Backtrace:
 0: (SB-KERNEL:TWO-ARG->= 0 NIL)
 1: (GIR::MAP-C-ARRAY #<FUNCTION (LAMBDA (GIR::POS) :IN GIR::MEM-GET) {10131D9CBB}> #.(SB-SYS:INT-SAP #X7F1EC80EBBD8) #<GIR::C-ARRAY-TYPE {10131E0FA3}>)
      Locals:
        ARRAY = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8)
        C-ARRAY-TYPE = #<GIR::C-ARRAY-TYPE {10131E0FA3}>
        FUNC = #<FUNCTION (LAMBDA (GIR::POS) :IN GIR::MEM-GET) {10131D9CBB}>
        I = 0
        LENGTH = NIL
        PARAM-SIZE = 1
        POS = #.(SB-SYS:INT-SAP #X7F1EC80EBBD8)
 2: ((:METHOD GIR::MEM-GET (T GIR::POINTER-TYPE)) #.(SB-SYS:INT-SAP #X7F1F97F7FFF8) #<GIR::POINTER-TYPE {10131E0FF3}>) [fast-method]
      Locals:
        GIR::POS = #.(SB-SYS:INT-SAP #X7F1F97F7FFF8)
        TYPE = #<GIR::POINTER-TYPE {10131E0FF3}>
 3: (GIR.FIELD:GET #.(SB-SYS:INT-SAP #X7F1ECC0036F0) #<GIR:FIELD-INFO {10087D1133}>)
 4: (PROCESS-FRAME #<GIR::STRUCT-INSTANCE {10131E02D3}>)
 5: ((LAMBDA (APPSINK) :IN START-PROCESSING) #<GIR::OBJECT-INSTANCE {10131E01B3}>)
 6: ((LAMBDA (GIR::CLOSURE RETURN GIR::N-VALUES GIR::PARAMS GIR::HINT GIR::DATA) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") #.(SB-SYS:INT-SAP #X7F1FCC470B30) #.(SB-S..
 7: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/nix/store/hijk9y2lcv24s3a5kv56cyxqcn3frgs4-source-patched/src/signal.lisp") #<unavailable argument> #<unavailable argument> #<..
 8: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
 9: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN))
10: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
11: ((FLET "WITHOUT-INTERRUPTS-BODY-" :IN SB-THREAD::RUN))
12: (SB-THREAD::RUN)
13: (SB-THREAD::ENTER-FOREIGN-CALLBACK 87 69885224809132 69885224809124)
14: ("foreign function: call_into_lisp_")
15: ("foreign function: funcall3")
16: ("foreign function: callback_wrapper_trampoline")
17: ("foreign function: #x501053AB")
 --more--

the source:

( in the git : https://git.sunshine.industries/efim/scratch-screen-share/src/commit/73316581a056a20e51dc36eb96673acfee7d90b0/gstreamer/capturing-frames.lisp )

;(load (sb-ext:posix-getenv "ASDF"))
(asdf:load-system 'cl-cffi-gtk)
(asdf:load-system 'cl-gobject-introspection)

(defpackage :capturing-frames
				(:use :cl))

(in-package :capturing-frames)

(defvar *gstreamer* (gir:require-namespace "Gst"))
;(defvar *gstapp* (gir:require-namespace "GstApp"))

(defun cast-with-videobox (&optional stream-num)
  (gir:invoke (*gstreamer* 'init) '())
  (let* ((pipeline (gir:invoke (*gstreamer* "Pipeline" 'new) "video-pipeline"))
         (source (gir:invoke (*gstreamer* "ElementFactory" 'make) (if stream-num
																							"pipewiresrc"
																							"videotestsrc") "source"))
         (convert (gir:invoke (*gstreamer* "ElementFactory" 'make) "videoconvert" "convert"))
			(tee (gir:invoke (*gstreamer* "ElementFactory" 'make) "tee" "tee"))
			(queue1 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue1"))
         (queue2 (gir:invoke (*gstreamer* "ElementFactory" 'make) "queue" "queue2"))
         (sink (gir:invoke (*gstreamer* "ElementFactory" 'make) "autovideosink" "sink"))
			(appsink (gir:invoke (*gstreamer* "ElementFactory" 'make) "appsink" "appsink"))
			)
	 (when stream-num
		;; Set pipewiresrc properties, such as the path
		(setf (gir:property source 'path) (format nil "~A" stream-num)))

	 ;; Add and link elements in the pipeline
    (gir:invoke (pipeline 'add) source)
    (gir:invoke (pipeline 'add) convert)
	 (gir:invoke (pipeline 'add) tee)
	 (gir:invoke (pipeline 'add) queue1)
	 (gir:invoke (pipeline 'add) queue2)
	 (gir:invoke (pipeline 'add) sink)
	 (gir:invoke (pipeline 'add) appsink)
    
    (gir:invoke (source 'link) convert)
	 (gir:invoke (convert 'link) tee)
	 (gir:invoke (tee 'link) queue1)
	 (gir:invoke (tee 'link) queue2)
    (gir:invoke (queue1 'link) sink)
	 (gir:invoke (queue2 'link) appsink)

    ;; Configure appsink
	 (setf (gir:property appsink "emit-signals") t
			 (gir:property appsink "max-buffers") 1
			 (gir:property appsink "drop") t)
    	 
	 (gir:invoke (pipeline 'set-state) (gir:nget *gstreamer* "State" :playing))
	 	 
	 (values pipeline appsink)))

(defun process-frame (sample)
  (format t ">>> got to process sample with ~A~%" sample)
  (let* ((buffer (gir:invoke (sample 'get-buffer)))
         (info (gir:invoke (sample 'get-info)))
			(caps (gir:invoke (sample 'get-caps)))
			(caps-struct (gir:invoke (caps 'get-structure) 0))
			(width (gir:invoke ((gir:invoke (caps-struct 'get-value) "width") 'get-int)))
			(height (gir:invoke ((gir:invoke (caps-struct 'get-value) "height") 'get-int))))
	 
	 (multiple-value-bind (found map-info) (gir:invoke (buffer 'map) (gir:nget *gstreamer* "MapFlags" :read))
		(format t ">>>> the buffer has offset ~A~%" (gir:field buffer 'offset))
		(format t ">>>> what's in struct? ~A~%" (gir:invoke (caps-struct 'to-string)))
		(format t ">>>> the fields are width ~A, height ~A~%" width height)
		(format t ">>>> and map result are ~A and ~A~%" found map-info)
		(when found
		  (let ((data (gir:field map-info 'data))
				  (size (gir:field map-info 'size)))
			 (format t ">>>> and struct info? ~A~%" map-info)
			 (gir:invoke (buffer 'unmap) map-info)
			 (format t ">>>> and data has size ~A and data is ~A~%" size (or data "empty data")))))))

(defun start-processing (appsink)
  (gir:connect appsink "new-sample"
               (lambda (appsink)
					  (format t ">> got printed object ~A~%" appsink)
					  (format t ">> got printed class ~A~%" (gir:gir-class-of appsink))
					  (format t ">> trying to get method on sink ~A~%" (gir:nget appsink 'pull-sample))
					  (let ((sample (gir:invoke (appsink 'pull-sample))))
                   (when sample
                     (process-frame sample)))

					  
                 (gir:nget *gstreamer* "FlowReturn" :ok))))

(defun try-run ()
  (multiple-value-bind (pipeline appsink) (cast-with-videobox)
	 (format t "got appsink ~A" appsink)
	 (start-processing appsink)
	 (loop
		(sleep 1))))

;(try-run)

submitted

https://github.com/andy128k/cl-gobject-introspection/issues/102

what's next?

i could try another library for gir bindings. not sure whether that would help much, but yeah

or i could just try to replicate this in python or maybe in go?

whelp gir2cl is also on top of cl-gobject-introspection

huh. let's set up a directory with go stuff? i guess

trying in separate directory with https://github.com/linuxdeepin/go-gir

but more realistically https://pkg.go.dev/github.com/go-gst/go-gst

this is very much different, but yeah. let's go with appsink example https://pkg.go.dev/github.com/go-gst/go-gst@v1.1.0/examples/appsink

for the go library to pick up gstreamer from nix shell https://discourse.nixos.org/t/flakes-how-to-make-pkg-config-work-with-nix-shell/16305 it has to somehow be exposed via pkg-config

yes, adding pkg-config to shell works, nice

[2024-08-24 Sat]

ok, continuing with go thing. into what do i want to decode stuff?

well, the reported default format is sample info: video/x-raw, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, format=(string)YV12, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive http://www.avisynth.nl/index.php/FAQ_YV12

but maybe i want something else?

let's read docs on appsrc https://github.com/go-gst/go-gst/tree/v1.1.0/examples/appsrc

[2024-12-08 Sun]

well, i guess i will maybe return to this?

https://www.youtube.com/watch?v=Jn_09BeUEI0 here is gltransformation this looks like what i'd really want, no?

and this is documentation

https://gstreamer.freedesktop.org/documentation/opengl/index.html?gi-language=c

https://gstreamer.freedesktop.org/documentation/opengl/gltransformation.html?gi-language=c#gltransformation why doesn't it work?

[2024-12-21 Sat]

wait. it works now? how and why

maybe because it's installed via apt? yes. if i try to use gst-launch-1.0 gltestsrc ! gltransformation scale-y=0.5 scale-x=0.5 ! glimagesink in the nix shell i get

Setting pipeline to PAUSED ...
Caught SIGSEGV
Spinning.  Please run 'gdb gst-launch-1.0 242683' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

for now found article about other ways to transform

https://lubosz.wordpress.com/2014/06/16/transforming-video-on-the-gpu/ but that this opengl way is much faster

comparing with this: https://gstreamer.freedesktop.org/documentation/videoconvertscale/videoscale.html?gi-language=c gst-launch-1.0 -v videotestsrc ! videoscale ! autovideosink

gst-launch-1.0 -v videotestsrc ! videoscale ! video/x-raw,width=1000 ! autovideosink

the ordinary error is

gst-launch-1.0 gltestsrc ! gltransformation rotation-z=45 ! glimagesink Setting pipeline to PAUSED … Caught SIGSEGV Spinning. Please run 'gdb gst-launch-1.0 7843' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

huh. trying previous nixpkgs releases

and in 23.05 the error is different efim-nefedov@LLF33A87M:~/Documents/personal/scratch-screen-share$ gst-launch-1.0 gltestsrc ! gltransformation rotation-z=45 ! glimagesink Setting pipeline to PAUSED … ERROR: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: Could not find any FBConfig's to use (check attributes?) Additional debug info: ../ext/gl/gstglimagesink.c(1131): _ensure_gl_setup (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink ERROR: pipeline doesn't want to preroll. Failed to set pipeline to PAUSED. Setting pipeline to NULL … Freeing pipeline …

ok, looking at the error the problem is in creating connection

WARN gldisplayegl gstgldisplay_egl.c:408:gst_gl_display_egl_from_gl_display:<gldisplayegl1> failed to get EGLDisplay from native display

but beats me if i know what needs to be done claude.ai recommends adding LD_LIBRARY_PATH to gstreamer so trying

        gstWrapper = pkgs.writeScriptBin "gst-launch-wrapped" ''
          #!${pkgs.bash}/bin/bash
              export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
              exec ${pkgs.gst_all_1.gstreamer}/bin/gst-launch-1.0 "$@"
        '';

but that fails $ gst-launch-wrapped gltestsrc ! gltransformation rotation-z=45 ! glimagesink /nix/store/p6k7xp1lsfmbdd731mlglrdj2d66mr82-bash-5.2p37/bin/bash: error while loading shared libraries: __vdso_gettimeofday: invalid mode for dlopen(): Invalid argument

[2024-12-22 Sun]

ok, let's try what

let's try default video out after i turn on the screen sharing from cl part

with code cleaned up i can start the share, that's nice

gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! autovideosink

let's also try to add opengl processing gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! gltransformation rotation-z=45 ! glimagesink

so, this kind of works gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-z=45 ! glimagesink

even if the thing is small

trying to figure out why gl things open a small window

one that works

GST_DEBUG=4,caps:5 gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! autovideosink has deterministic way of creating a stream-id 0:00:00.032812186 44186 0x74b020000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw, format=(string)BGRA, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1 0:00:00.033493070 44186 0x74b020000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)YV12

and comparing with

GST_DEBUG=4,caps:5 gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-z=45 ! glimagesink (by recommendation of claude) 0:00:00.152714238 44533 0x784a70000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)RGBA, texture-target=(string)2D 0:00:00.152789062 44533 0x784a70000b70 INFO basetransform gstbasetransform.c:1326:gst_base_transform_setcaps:<glcolorbalance0> reuse caps 0:00:00.152821256 44533 0x784a70000b70 INFO GST_EVENT gstevent.c:918:gst_event_new_caps: creating caps event video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, framerate=(fraction)0/1, max-framerate=(fraction)60/1, format=(string)RGBA, texture-target=(string)2D

well, i guess i can just open the window full size

ok! looks like it does stream full resolution.

now. i'd want to open the displaying pipeline programmatically and then maybe try to dynamically change params

i guess even without a window embedding, just starting a window is good enough but also, hm.

do i need any ui for the video window? i really don't think so. maybe let's figure out the missing cursor gst-launch-1.0 pipewiresrc path=90 ! videoconvert ! capsfilter caps="video/x-raw,format=RGBA" ! glupload ! gltransformation rotation-x=-15 ortho=True ! glimagesink gst-launch-1.0 gltestsrc ! gltransformation rotation-x=-45 ortho=True ! glimagesink

am i not getting into Create Session? after the Start?

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.ScreenCast.html (also note that here are cursor options)

33 KiB Raw Permalink Blame History

[2024-06-28 Fri]

well, i'm setting up sbcl and sly?

well, just flake shell and (use-package sly) is enough

[2024-06-29 Sat]

going through the dbus guide, sounds neat

now let's read about calling methods

the receiving of the signals

all the different tabs i had

overall freedesktop portal dbus api docs

searching github for 'get-managed-objects usage

searchin github for 'define-singlan-handler usage examples

example in cl-notify of calling a method 'with-introspected-object